2023-07-30 23:55:24,080 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4 2023-07-30 23:55:24,094 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-log-dir so I do NOT create it in target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192 2023-07-30 23:55:24,094 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-log-dir Erasing configuration value by system value. 2023-07-30 23:55:24,094 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-tmp-dir so I do NOT create it in target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192 2023-07-30 23:55:24,095 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-30 23:55:24,095 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192 2023-07-30 23:55:24,114 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-30 23:55:24,115 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL timeout: 13 mins 2023-07-30 23:55:24,153 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=171, AvailableMemoryMB=2209 2023-07-30 23:55:24,168 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-log-dir so I do NOT create it in target/test-data/89dbb39d-bb63-12b3-0ddd-87e152f08d18 2023-07-30 23:55:24,168 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-log-dir Erasing configuration value by system value. 2023-07-30 23:55:24,169 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-tmp-dir so I do NOT create it in target/test-data/89dbb39d-bb63-12b3-0ddd-87e152f08d18 2023-07-30 23:55:24,169 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-30 23:55:24,169 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/89dbb39d-bb63-12b3-0ddd-87e152f08d18 2023-07-30 23:55:24,178 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-log-dir so I do NOT create it in target/test-data/03a2c8ae-5220-c13a-4b8c-84fc624cad03 2023-07-30 23:55:24,178 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-log-dir Erasing configuration value by system value. 2023-07-30 23:55:24,179 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-tmp-dir so I do NOT create it in target/test-data/03a2c8ae-5220-c13a-4b8c-84fc624cad03 2023-07-30 23:55:24,179 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-30 23:55:24,179 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/03a2c8ae-5220-c13a-4b8c-84fc624cad03 2023-07-30 23:55:24,183 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478, deleteOnExit=true 2023-07-30 23:55:24,303 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/zookeeper_0, clientPort=52585, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-30 23:55:24,321 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52585 2023-07-30 23:55:24,325 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-30 23:55:24,325 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-30 23:55:24,326 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/test.cache.data in system properties and HBase conf 2023-07-30 23:55:24,326 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop.tmp.dir in system properties and HBase conf 2023-07-30 23:55:24,327 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/hadoop.log.dir in system properties and HBase conf 2023-07-30 23:55:24,327 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-30 23:55:24,327 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-30 23:55:24,328 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-30 23:55:24,442 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-30 23:55:24,857 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-30 23:55:24,862 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-30 23:55:24,862 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-30 23:55:24,862 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-30 23:55:24,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-30 23:55:24,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-30 23:55:24,865 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-30 23:55:24,865 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-30 23:55:24,865 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-30 23:55:24,865 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-30 23:55:24,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/nfs.dump.dir in system properties and HBase conf 2023-07-30 23:55:24,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/java.io.tmpdir in system properties and HBase conf 2023-07-30 23:55:24,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-30 23:55:24,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-30 23:55:24,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-30 23:55:25,396 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-30 23:55:25,400 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-30 23:55:25,688 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-30 23:55:25,818 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-30 23:55:25,833 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:25,868 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:25,900 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/java.io.tmpdir/Jetty_localhost_38487_hdfs____.3iao4o/webapp 2023-07-30 23:55:26,028 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38487 2023-07-30 23:55:26,045 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-30 23:55:26,046 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-30 23:55:26,561 WARN [Listener at localhost/45087] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:26,630 WARN [Listener at localhost/45087] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-30 23:55:26,651 WARN [Listener at localhost/45087] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:26,658 INFO [Listener at localhost/45087] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:26,662 INFO [Listener at localhost/45087] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/java.io.tmpdir/Jetty_localhost_33291_datanode____67zxox/webapp 2023-07-30 23:55:26,772 INFO [Listener at localhost/45087] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33291 2023-07-30 23:55:27,079 WARN [Listener at localhost/36177] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:27,091 WARN [Listener at localhost/36177] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-30 23:55:27,094 WARN [Listener at localhost/36177] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:27,095 INFO [Listener at localhost/36177] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:27,107 INFO [Listener at localhost/36177] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/java.io.tmpdir/Jetty_localhost_44707_datanode____.m4x9uj/webapp 2023-07-30 23:55:27,221 INFO [Listener at localhost/36177] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44707 2023-07-30 23:55:27,234 WARN [Listener at localhost/45953] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:27,561 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe4541f72d01fd590: Processing first storage report for DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f from datanode e5120aa3-5268-4e18-b3d5-adf3dedbc59f 2023-07-30 23:55:27,563 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe4541f72d01fd590: from storage DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f node DatanodeRegistration(127.0.0.1:37233, datanodeUuid=e5120aa3-5268-4e18-b3d5-adf3dedbc59f, infoPort=42815, infoSecurePort=0, ipcPort=36177, storageInfo=lv=-57;cid=testClusterID;nsid=2090517051;c=1690761325473), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-30 23:55:27,563 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5bfc2f04922cbd4a: Processing first storage report for DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde from datanode 2759b877-2b67-4694-a815-b33dbea45ff5 2023-07-30 23:55:27,564 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5bfc2f04922cbd4a: from storage DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde node DatanodeRegistration(127.0.0.1:42315, datanodeUuid=2759b877-2b67-4694-a815-b33dbea45ff5, infoPort=44709, infoSecurePort=0, ipcPort=45953, storageInfo=lv=-57;cid=testClusterID;nsid=2090517051;c=1690761325473), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:27,564 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe4541f72d01fd590: Processing first storage report for DS-96b6fde7-3145-4381-9140-967807a839a4 from datanode e5120aa3-5268-4e18-b3d5-adf3dedbc59f 2023-07-30 23:55:27,564 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe4541f72d01fd590: from storage DS-96b6fde7-3145-4381-9140-967807a839a4 node DatanodeRegistration(127.0.0.1:37233, datanodeUuid=e5120aa3-5268-4e18-b3d5-adf3dedbc59f, infoPort=42815, infoSecurePort=0, ipcPort=36177, storageInfo=lv=-57;cid=testClusterID;nsid=2090517051;c=1690761325473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:27,564 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5bfc2f04922cbd4a: Processing first storage report for DS-6c63ca6e-6d44-4381-86e9-bbac3006977f from datanode 2759b877-2b67-4694-a815-b33dbea45ff5 2023-07-30 23:55:27,564 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5bfc2f04922cbd4a: from storage DS-6c63ca6e-6d44-4381-86e9-bbac3006977f node DatanodeRegistration(127.0.0.1:42315, datanodeUuid=2759b877-2b67-4694-a815-b33dbea45ff5, infoPort=44709, infoSecurePort=0, ipcPort=45953, storageInfo=lv=-57;cid=testClusterID;nsid=2090517051;c=1690761325473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:27,657 DEBUG [Listener at localhost/45953] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4 2023-07-30 23:55:27,667 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:27,669 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:28,328 INFO [Listener at localhost/45953] util.FSUtils(471): Created version file at hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5 with version=8 2023-07-30 23:55:28,328 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/hbase-staging 2023-07-30 23:55:28,659 INFO [Listener at localhost/45953] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-30 23:55:29,166 INFO [Listener at localhost/45953] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-30 23:55:29,207 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,208 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,208 INFO [Listener at localhost/45953] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:29,209 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,209 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:29,351 INFO [Listener at localhost/45953] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:29,423 DEBUG [Listener at localhost/45953] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-30 23:55:29,519 INFO [Listener at localhost/45953] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43527 2023-07-30 23:55:29,529 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,531 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,551 INFO [Listener at localhost/45953] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43527 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:29,593 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:435270x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:29,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43527-0x101b8a8e9d90000 connected 2023-07-30 23:55:29,618 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:55:29,619 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:55:29,622 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-30 23:55:29,633 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43527 2023-07-30 23:55:29,634 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43527 2023-07-30 23:55:29,634 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43527 2023-07-30 23:55:29,635 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43527 2023-07-30 23:55:29,635 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43527 2023-07-30 23:55:29,640 INFO [Listener at localhost/45953] master.HMaster(444): hbase.rootdir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5, hbase.cluster.distributed=false 2023-07-30 23:55:29,725 INFO [Listener at localhost/45953] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-30 23:55:29,726 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,726 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,726 INFO [Listener at localhost/45953] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:29,726 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,726 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:29,731 INFO [Listener at localhost/45953] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:29,734 INFO [Listener at localhost/45953] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42313 2023-07-30 23:55:29,736 INFO [Listener at localhost/45953] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:29,742 DEBUG [Listener at localhost/45953] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:29,743 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,746 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,747 INFO [Listener at localhost/45953] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42313 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:29,751 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:423130x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:29,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42313-0x101b8a8e9d90001 connected 2023-07-30 23:55:29,752 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:55:29,753 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:55:29,754 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-30 23:55:29,754 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42313 2023-07-30 23:55:29,754 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42313 2023-07-30 23:55:29,755 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42313 2023-07-30 23:55:29,755 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42313 2023-07-30 23:55:29,756 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42313 2023-07-30 23:55:29,770 INFO [Listener at localhost/45953] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-30 23:55:29,771 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,771 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,772 INFO [Listener at localhost/45953] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:29,772 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:29,772 INFO [Listener at localhost/45953] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:29,772 INFO [Listener at localhost/45953] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:29,774 INFO [Listener at localhost/45953] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36237 2023-07-30 23:55:29,774 INFO [Listener at localhost/45953] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:29,776 DEBUG [Listener at localhost/45953] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:29,777 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,778 INFO [Listener at localhost/45953] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,780 INFO [Listener at localhost/45953] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36237 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:29,785 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:362370x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:29,786 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): regionserver:362370x0, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:55:29,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36237-0x101b8a8e9d90002 connected 2023-07-30 23:55:29,787 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:55:29,787 DEBUG [Listener at localhost/45953] zookeeper.ZKUtil(164): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-30 23:55:29,788 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36237 2023-07-30 23:55:29,788 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36237 2023-07-30 23:55:29,788 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36237 2023-07-30 23:55:29,789 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36237 2023-07-30 23:55:29,789 DEBUG [Listener at localhost/45953] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36237 2023-07-30 23:55:29,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:29,799 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-30 23:55:29,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:29,821 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-30 23:55:29,821 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-30 23:55:29,821 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-30 23:55:29,821 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:29,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-30 23:55:29,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,43527,1690761328479 from backup master directory 2023-07-30 23:55:29,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-30 23:55:29,827 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:29,827 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-30 23:55:29,827 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:29,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:29,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-30 23:55:29,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-30 23:55:29,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/hbase.id with ID: c047cc16-085a-45ee-a021-aa0b6e22ca94 2023-07-30 23:55:29,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:29,971 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:30,012 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2e15bf89 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:30,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4bf8f6ba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:30,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:30,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-30 23:55:30,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:30,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:30,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store-tmp 2023-07-30 23:55:30,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:30,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:55:30,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:30,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:30,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:55:30,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:30,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:30,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:30,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:30,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:30,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/oldWALs, maxLogs=10 2023-07-30 23:55:30,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-30 23:55:30,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-30 23:55:30,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-30 23:55:30,253 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-30 23:55:30,255 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:55:30,255 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:55:30,262 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-30 23:55:30,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 2023-07-30 23:55:30,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:55:30,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:30,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:30,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:30,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:30,390 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:30,398 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-30 23:55:30,420 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-30 23:55:30,432 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:30,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:30,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:30,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:30,459 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:30,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62975769, jitterRate=-0.06158791482448578}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:55:30,460 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:30,461 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-30 23:55:30,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-30 23:55:30,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-30 23:55:30,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-30 23:55:30,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-30 23:55:30,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 41 msec 2023-07-30 23:55:30,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-30 23:55:30,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-30 23:55:30,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-30 23:55:30,584 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-30 23:55:30,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-30 23:55:30,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-30 23:55:30,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-30 23:55:30,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-30 23:55:30,601 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:30,602 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-30 23:55:30,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-30 23:55:30,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-30 23:55:30,619 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-30 23:55:30,619 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-30 23:55:30,619 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-30 23:55:30,619 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:30,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,43527,1690761328479, sessionid=0x101b8a8e9d90000, setting cluster-up flag (Was=false) 2023-07-30 23:55:30,635 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:30,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-30 23:55:30,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:30,647 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:30,653 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-30 23:55:30,654 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:30,656 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.hbase-snapshot/.tmp 2023-07-30 23:55:30,693 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(951): ClusterId : c047cc16-085a-45ee-a021-aa0b6e22ca94 2023-07-30 23:55:30,693 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(951): ClusterId : c047cc16-085a-45ee-a021-aa0b6e22ca94 2023-07-30 23:55:30,698 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:30,698 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:30,704 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:30,704 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:30,704 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:30,704 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:30,710 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:30,710 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:30,712 DEBUG [RS:0;jenkins-hbase4:42313] zookeeper.ReadOnlyZKClient(139): Connect 0x2e7a8c53 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:30,712 DEBUG [RS:1;jenkins-hbase4:36237] zookeeper.ReadOnlyZKClient(139): Connect 0x443a81ac to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:30,717 DEBUG [RS:0;jenkins-hbase4:42313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20bd77ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:30,717 DEBUG [RS:1;jenkins-hbase4:36237] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4afd675b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:30,718 DEBUG [RS:0;jenkins-hbase4:42313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2482e723, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:30,718 DEBUG [RS:1;jenkins-hbase4:36237] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ec2eadc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:30,748 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42313 2023-07-30 23:55:30,750 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:36237 2023-07-30 23:55:30,753 INFO [RS:0;jenkins-hbase4:42313] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:30,753 INFO [RS:1;jenkins-hbase4:36237] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:30,754 INFO [RS:0;jenkins-hbase4:42313] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:30,754 INFO [RS:1;jenkins-hbase4:36237] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:30,754 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:30,754 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:30,757 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43527,1690761328479 with isa=jenkins-hbase4.apache.org/172.31.14.131:36237, startcode=1690761329770 2023-07-30 23:55:30,757 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43527,1690761328479 with isa=jenkins-hbase4.apache.org/172.31.14.131:42313, startcode=1690761329724 2023-07-30 23:55:30,775 DEBUG [RS:0;jenkins-hbase4:42313] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:30,775 DEBUG [RS:1;jenkins-hbase4:36237] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:30,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:30,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:30,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:30,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:30,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:30,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-30 23:55:30,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:30,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690761360790 2023-07-30 23:55:30,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-30 23:55:30,807 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-30 23:55:30,811 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:30,812 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-30 23:55:30,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-30 23:55:30,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-30 23:55:30,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-30 23:55:30,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-30 23:55:30,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,817 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:30,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-30 23:55:30,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-30 23:55:30,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-30 23:55:30,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-30 23:55:30,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-30 23:55:30,829 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761330825,5,FailOnTimeoutGroup] 2023-07-30 23:55:30,829 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761330829,5,FailOnTimeoutGroup] 2023-07-30 23:55:30,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-30 23:55:30,831 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44159, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:30,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,831 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43673, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:30,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,843 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43527] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:30,859 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:30,860 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:30,860 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5 2023-07-30 23:55:30,861 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43527] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:30,868 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5 2023-07-30 23:55:30,868 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5 2023-07-30 23:55:30,868 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45087 2023-07-30 23:55:30,868 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45087 2023-07-30 23:55:30,868 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:30,868 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:30,875 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:55:30,876 DEBUG [RS:1;jenkins-hbase4:36237] zookeeper.ZKUtil(162): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:30,876 DEBUG [RS:0;jenkins-hbase4:42313] zookeeper.ZKUtil(162): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:30,876 WARN [RS:1;jenkins-hbase4:36237] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:30,877 WARN [RS:0;jenkins-hbase4:42313] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:30,878 INFO [RS:1;jenkins-hbase4:36237] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:30,878 INFO [RS:0;jenkins-hbase4:42313] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:30,878 INFO [RS:1;jenkins-hbase4:36237] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:30,878 INFO [RS:0;jenkins-hbase4:42313] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:30,879 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:30,880 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:30,880 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36237,1690761329770] 2023-07-30 23:55:30,880 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:30,884 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:30,889 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:55:30,891 DEBUG [RS:0;jenkins-hbase4:42313] zookeeper.ZKUtil(162): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:30,891 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info 2023-07-30 23:55:30,892 DEBUG [RS:0;jenkins-hbase4:42313] zookeeper.ZKUtil(162): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:30,892 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:55:30,893 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:30,894 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:55:30,896 DEBUG [RS:1;jenkins-hbase4:36237] zookeeper.ZKUtil(162): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:30,897 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:30,897 DEBUG [RS:1;jenkins-hbase4:36237] zookeeper.ZKUtil(162): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:30,898 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:55:30,899 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:30,899 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:55:30,903 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table 2023-07-30 23:55:30,904 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:55:30,905 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:30,906 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:30,906 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:30,907 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740 2023-07-30 23:55:30,908 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740 2023-07-30 23:55:30,913 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:55:30,915 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:55:30,916 INFO [RS:0;jenkins-hbase4:42313] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:30,916 INFO [RS:1;jenkins-hbase4:36237] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:30,920 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:30,922 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69121104, jitterRate=0.029984712600708008}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:55:30,922 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:55:30,922 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:55:30,922 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:55:30,922 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:55:30,922 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:55:30,922 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:55:30,923 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:55:30,924 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:55:30,931 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:30,931 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-30 23:55:30,943 INFO [RS:1;jenkins-hbase4:36237] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:30,943 INFO [RS:0;jenkins-hbase4:42313] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:30,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-30 23:55:30,948 INFO [RS:1;jenkins-hbase4:36237] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:30,948 INFO [RS:0;jenkins-hbase4:42313] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:30,949 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,949 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,950 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:30,954 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:30,961 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,961 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,961 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,961 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,962 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-30 23:55:30,962 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,962 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,962 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,963 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,963 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:30,964 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:30,964 DEBUG [RS:1;jenkins-hbase4:36237] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,964 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,965 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,965 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,965 DEBUG [RS:0;jenkins-hbase4:42313] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:30,967 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-30 23:55:30,968 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,968 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,968 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,971 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,971 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,971 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,992 INFO [RS:1;jenkins-hbase4:36237] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:30,992 INFO [RS:0;jenkins-hbase4:42313] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:30,995 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42313,1690761329724-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:30,995 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36237,1690761329770-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:31,014 INFO [RS:0;jenkins-hbase4:42313] regionserver.Replication(203): jenkins-hbase4.apache.org,42313,1690761329724 started 2023-07-30 23:55:31,014 INFO [RS:1;jenkins-hbase4:36237] regionserver.Replication(203): jenkins-hbase4.apache.org,36237,1690761329770 started 2023-07-30 23:55:31,014 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42313,1690761329724, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42313, sessionid=0x101b8a8e9d90001 2023-07-30 23:55:31,014 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36237,1690761329770, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36237, sessionid=0x101b8a8e9d90002 2023-07-30 23:55:31,014 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:31,014 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:31,014 DEBUG [RS:0;jenkins-hbase4:42313] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:31,014 DEBUG [RS:1;jenkins-hbase4:36237] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:31,015 DEBUG [RS:0;jenkins-hbase4:42313] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42313,1690761329724' 2023-07-30 23:55:31,015 DEBUG [RS:1;jenkins-hbase4:36237] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36237,1690761329770' 2023-07-30 23:55:31,015 DEBUG [RS:1;jenkins-hbase4:36237] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-30 23:55:31,015 DEBUG [RS:0;jenkins-hbase4:42313] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-30 23:55:31,016 DEBUG [RS:0;jenkins-hbase4:42313] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-30 23:55:31,016 DEBUG [RS:1;jenkins-hbase4:36237] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-30 23:55:31,017 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:31,017 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:31,017 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:31,017 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:31,017 DEBUG [RS:0;jenkins-hbase4:42313] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:31,017 DEBUG [RS:1;jenkins-hbase4:36237] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:31,017 DEBUG [RS:0;jenkins-hbase4:42313] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42313,1690761329724' 2023-07-30 23:55:31,018 DEBUG [RS:0;jenkins-hbase4:42313] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-30 23:55:31,017 DEBUG [RS:1;jenkins-hbase4:36237] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36237,1690761329770' 2023-07-30 23:55:31,018 DEBUG [RS:1;jenkins-hbase4:36237] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-30 23:55:31,018 DEBUG [RS:1;jenkins-hbase4:36237] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-30 23:55:31,018 DEBUG [RS:0;jenkins-hbase4:42313] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-30 23:55:31,019 DEBUG [RS:1;jenkins-hbase4:36237] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:31,019 DEBUG [RS:0;jenkins-hbase4:42313] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:31,019 INFO [RS:1;jenkins-hbase4:36237] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:31,019 INFO [RS:0;jenkins-hbase4:42313] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:31,019 INFO [RS:1;jenkins-hbase4:36237] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:31,019 INFO [RS:0;jenkins-hbase4:42313] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:31,120 DEBUG [jenkins-hbase4:43527] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-30 23:55:31,123 DEBUG [jenkins-hbase4:43527] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:31,128 INFO [RS:1;jenkins-hbase4:36237] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:31,128 INFO [RS:0;jenkins-hbase4:42313] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:31,130 DEBUG [jenkins-hbase4:43527] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:31,130 DEBUG [jenkins-hbase4:43527] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:31,130 DEBUG [jenkins-hbase4:43527] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:31,132 INFO [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:55:31,132 INFO [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:55:31,133 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42313,1690761329724, state=OPENING 2023-07-30 23:55:31,148 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-30 23:55:31,150 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:31,151 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-30 23:55:31,158 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:55:31,158 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42313,1690761329724}] 2023-07-30 23:55:31,159 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:55:31,160 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:55:31,160 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:55:31,168 INFO [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 2023-07-30 23:55:31,168 DEBUG [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:55:31,171 INFO [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 2023-07-30 23:55:31,172 DEBUG [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:55:31,348 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:31,350 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:31,353 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32830, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:31,365 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-30 23:55:31,366 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:31,366 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:31,366 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:31,369 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42313%2C1690761329724.meta, suffix=.meta, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:55:31,386 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:55:31,388 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:55:31,394 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.meta.1690761331371.meta 2023-07-30 23:55:31,395 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:55:31,396 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:31,399 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-30 23:55:31,418 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-30 23:55:31,454 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-30 23:55:31,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-30 23:55:31,460 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:31,460 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-30 23:55:31,460 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-30 23:55:31,463 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:55:31,465 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info 2023-07-30 23:55:31,466 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info 2023-07-30 23:55:31,467 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:55:31,468 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:31,468 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:55:31,469 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:31,469 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:31,470 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:55:31,471 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:31,471 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:55:31,472 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table 2023-07-30 23:55:31,472 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table 2023-07-30 23:55:31,473 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:55:31,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:31,475 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740 2023-07-30 23:55:31,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740 2023-07-30 23:55:31,481 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:55:31,484 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:55:31,485 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73944325, jitterRate=0.1018563061952591}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:55:31,485 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:55:31,496 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690761331340 2023-07-30 23:55:31,514 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-30 23:55:31,515 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-30 23:55:31,515 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42313,1690761329724, state=OPEN 2023-07-30 23:55:31,518 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-30 23:55:31,518 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-30 23:55:31,522 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-30 23:55:31,522 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42313,1690761329724 in 360 msec 2023-07-30 23:55:31,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-30 23:55:31,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 580 msec 2023-07-30 23:55:31,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 829 msec 2023-07-30 23:55:31,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690761331532, completionTime=-1 2023-07-30 23:55:31,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-30 23:55:31,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-30 23:55:31,594 DEBUG [hconnection-0x26d8c6c8-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:31,597 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32832, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:31,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-30 23:55:31,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690761391614 2023-07-30 23:55:31,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690761451614 2023-07-30 23:55:31,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 80 msec 2023-07-30 23:55:31,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43527,1690761328479-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:31,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43527,1690761328479-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:31,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43527,1690761328479-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:31,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43527, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:31,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:31,647 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-30 23:55:31,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-30 23:55:31,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:31,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-30 23:55:31,668 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-30 23:55:31,670 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-30 23:55:31,691 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:31,694 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 empty. 2023-07-30 23:55:31,694 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:31,694 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-30 23:55:31,725 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:31,727 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => ef4a62973dc7531db8626f8122ae55c2, NAME => 'hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp 2023-07-30 23:55:31,744 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:31,744 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing ef4a62973dc7531db8626f8122ae55c2, disabling compactions & flushes 2023-07-30 23:55:31,744 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:31,744 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:31,744 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. after waiting 0 ms 2023-07-30 23:55:31,744 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:31,744 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:31,744 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for ef4a62973dc7531db8626f8122ae55c2: 2023-07-30 23:55:31,752 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-30 23:55:31,768 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761331755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690761331755"}]},"ts":"1690761331755"} 2023-07-30 23:55:31,794 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-30 23:55:31,795 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-30 23:55:31,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761331795"}]},"ts":"1690761331795"} 2023-07-30 23:55:31,803 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-30 23:55:31,806 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:31,808 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:31,808 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:31,808 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:31,811 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN}] 2023-07-30 23:55:31,813 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN 2023-07-30 23:55:31,816 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42313,1690761329724; forceNewPlan=false, retain=false 2023-07-30 23:55:31,968 INFO [jenkins-hbase4:43527] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-30 23:55:31,969 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ef4a62973dc7531db8626f8122ae55c2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:31,969 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761331969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761331969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761331969"}]},"ts":"1690761331969"} 2023-07-30 23:55:31,975 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ef4a62973dc7531db8626f8122ae55c2, server=jenkins-hbase4.apache.org,42313,1690761329724}] 2023-07-30 23:55:32,136 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:32,136 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:32,140 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:55:32,156 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:55:32,157 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:55:32,161 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141 2023-07-30 23:55:32,162 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:55:32,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ef4a62973dc7531db8626f8122ae55c2, NAME => 'hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:32,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:32,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,166 INFO [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,169 DEBUG [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info 2023-07-30 23:55:32,169 DEBUG [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info 2023-07-30 23:55:32,169 INFO [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef4a62973dc7531db8626f8122ae55c2 columnFamilyName info 2023-07-30 23:55:32,170 INFO [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] regionserver.HStore(310): Store=ef4a62973dc7531db8626f8122ae55c2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:32,172 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,177 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:32,181 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:32,182 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ef4a62973dc7531db8626f8122ae55c2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72466692, jitterRate=0.0798378586769104}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:55:32,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ef4a62973dc7531db8626f8122ae55c2: 2023-07-30 23:55:32,184 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2., pid=6, masterSystemTime=1690761332128 2023-07-30 23:55:32,188 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:32,188 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:32,189 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ef4a62973dc7531db8626f8122ae55c2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:32,189 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761332188"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761332188"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761332188"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761332188"}]},"ts":"1690761332188"} 2023-07-30 23:55:32,196 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-30 23:55:32,196 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ef4a62973dc7531db8626f8122ae55c2, server=jenkins-hbase4.apache.org,42313,1690761329724 in 218 msec 2023-07-30 23:55:32,200 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-30 23:55:32,201 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN in 385 msec 2023-07-30 23:55:32,202 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-30 23:55:32,202 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761332202"}]},"ts":"1690761332202"} 2023-07-30 23:55:32,205 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-30 23:55:32,209 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-30 23:55:32,212 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 552 msec 2023-07-30 23:55:32,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-30 23:55:32,270 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-30 23:55:32,270 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:32,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-30 23:55:32,343 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-30 23:55:32,353 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 48 msec 2023-07-30 23:55:32,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-30 23:55:32,384 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-30 23:55:32,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 28 msec 2023-07-30 23:55:32,402 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-30 23:55:32,407 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-30 23:55:32,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.579sec 2023-07-30 23:55:32,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-30 23:55:32,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-30 23:55:32,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-30 23:55:32,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43527,1690761328479-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-30 23:55:32,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43527,1690761328479-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-30 23:55:32,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-30 23:55:32,501 DEBUG [Listener at localhost/45953] zookeeper.ReadOnlyZKClient(139): Connect 0x26a35432 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:32,512 DEBUG [Listener at localhost/45953] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@687c34c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:32,530 DEBUG [hconnection-0x74b287ff-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:32,545 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32838, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:32,556 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:32,556 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-30 23:55:32,556 INFO [Listener at localhost/45953] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5, deleteOnExit=true 2023-07-30 23:55:32,556 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-30 23:55:32,571 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/test.cache.data in system properties and HBase conf 2023-07-30 23:55:32,571 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/hadoop.tmp.dir in system properties and HBase conf 2023-07-30 23:55:32,571 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/hadoop.log.dir in system properties and HBase conf 2023-07-30 23:55:32,571 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-30 23:55:32,571 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-30 23:55:32,572 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-30 23:55:32,572 DEBUG [Listener at localhost/45953] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-30 23:55:32,572 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-30 23:55:32,572 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-30 23:55:32,572 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-30 23:55:32,573 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-30 23:55:32,573 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-30 23:55:32,573 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-30 23:55:32,573 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-30 23:55:32,573 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-30 23:55:32,573 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-30 23:55:32,574 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/nfs.dump.dir in system properties and HBase conf 2023-07-30 23:55:32,574 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/java.io.tmpdir in system properties and HBase conf 2023-07-30 23:55:32,574 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-30 23:55:32,574 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-30 23:55:32,574 INFO [Listener at localhost/45953] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-30 23:55:32,587 WARN [Listener at localhost/45953] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-30 23:55:32,587 WARN [Listener at localhost/45953] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-30 23:55:32,652 WARN [Listener at localhost/45953] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:32,654 INFO [Listener at localhost/45953] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:32,660 INFO [Listener at localhost/45953] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/java.io.tmpdir/Jetty_localhost_33669_hdfs____f5q2f7/webapp 2023-07-30 23:55:32,782 INFO [Listener at localhost/45953] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33669 2023-07-30 23:55:32,796 WARN [Listener at localhost/45953] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-30 23:55:32,796 WARN [Listener at localhost/45953] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-30 23:55:32,864 WARN [Listener at localhost/35081] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:32,882 WARN [Listener at localhost/35081] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-30 23:55:32,885 WARN [Listener at localhost/35081] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:32,886 INFO [Listener at localhost/35081] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:32,892 INFO [Listener at localhost/35081] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/java.io.tmpdir/Jetty_localhost_42199_datanode____.qpcn9i/webapp 2023-07-30 23:55:32,986 INFO [Listener at localhost/35081] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42199 2023-07-30 23:55:32,996 WARN [Listener at localhost/34151] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:33,016 WARN [Listener at localhost/34151] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-30 23:55:33,018 WARN [Listener at localhost/34151] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:33,020 INFO [Listener at localhost/34151] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:33,025 INFO [Listener at localhost/34151] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/java.io.tmpdir/Jetty_localhost_36527_datanode____e2jsda/webapp 2023-07-30 23:55:33,176 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfae67c3ed3be7f66: Processing first storage report for DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0 from datanode b6310d0f-aa61-4bd0-bc5a-06e0b38ddc49 2023-07-30 23:55:33,176 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfae67c3ed3be7f66: from storage DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0 node DatanodeRegistration(127.0.0.1:44771, datanodeUuid=b6310d0f-aa61-4bd0-bc5a-06e0b38ddc49, infoPort=46347, infoSecurePort=0, ipcPort=34151, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,176 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfae67c3ed3be7f66: Processing first storage report for DS-4c358560-e4ce-455c-800b-f82f25551ea5 from datanode b6310d0f-aa61-4bd0-bc5a-06e0b38ddc49 2023-07-30 23:55:33,177 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfae67c3ed3be7f66: from storage DS-4c358560-e4ce-455c-800b-f82f25551ea5 node DatanodeRegistration(127.0.0.1:44771, datanodeUuid=b6310d0f-aa61-4bd0-bc5a-06e0b38ddc49, infoPort=46347, infoSecurePort=0, ipcPort=34151, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,196 INFO [Listener at localhost/34151] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36527 2023-07-30 23:55:33,207 WARN [Listener at localhost/46013] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:33,230 WARN [Listener at localhost/46013] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-30 23:55:33,233 WARN [Listener at localhost/46013] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:33,234 INFO [Listener at localhost/46013] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:33,239 INFO [Listener at localhost/46013] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/java.io.tmpdir/Jetty_localhost_40023_datanode____.i50qas/webapp 2023-07-30 23:55:33,341 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x84729696f19871ca: Processing first storage report for DS-6286f008-7c89-4243-9f75-4f0cf659f3ae from datanode 355d7b89-5ff3-4b64-a399-067a26fed969 2023-07-30 23:55:33,342 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x84729696f19871ca: from storage DS-6286f008-7c89-4243-9f75-4f0cf659f3ae node DatanodeRegistration(127.0.0.1:34927, datanodeUuid=355d7b89-5ff3-4b64-a399-067a26fed969, infoPort=42711, infoSecurePort=0, ipcPort=46013, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,342 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x84729696f19871ca: Processing first storage report for DS-16c4f99b-c0b0-4fae-b46d-8a197ee22e89 from datanode 355d7b89-5ff3-4b64-a399-067a26fed969 2023-07-30 23:55:33,342 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x84729696f19871ca: from storage DS-16c4f99b-c0b0-4fae-b46d-8a197ee22e89 node DatanodeRegistration(127.0.0.1:34927, datanodeUuid=355d7b89-5ff3-4b64-a399-067a26fed969, infoPort=42711, infoSecurePort=0, ipcPort=46013, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,368 INFO [Listener at localhost/46013] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40023 2023-07-30 23:55:33,380 WARN [Listener at localhost/38149] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:33,410 WARN [Listener at localhost/38149] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-30 23:55:33,413 WARN [Listener at localhost/38149] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-30 23:55:33,414 INFO [Listener at localhost/38149] log.Slf4jLog(67): jetty-6.1.26 2023-07-30 23:55:33,419 INFO [Listener at localhost/38149] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/java.io.tmpdir/Jetty_localhost_33499_datanode____h9mwuz/webapp 2023-07-30 23:55:33,505 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4d7a1d55b7b6e393: Processing first storage report for DS-f652aaa1-b470-4fc3-9d06-0a0a18420452 from datanode a04145b2-4eb9-4c28-8495-692e417b0af2 2023-07-30 23:55:33,506 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4d7a1d55b7b6e393: from storage DS-f652aaa1-b470-4fc3-9d06-0a0a18420452 node DatanodeRegistration(127.0.0.1:40315, datanodeUuid=a04145b2-4eb9-4c28-8495-692e417b0af2, infoPort=45213, infoSecurePort=0, ipcPort=38149, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,506 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4d7a1d55b7b6e393: Processing first storage report for DS-4d9440f0-ab04-4ab7-838c-67a50312c088 from datanode a04145b2-4eb9-4c28-8495-692e417b0af2 2023-07-30 23:55:33,506 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4d7a1d55b7b6e393: from storage DS-4d9440f0-ab04-4ab7-838c-67a50312c088 node DatanodeRegistration(127.0.0.1:40315, datanodeUuid=a04145b2-4eb9-4c28-8495-692e417b0af2, infoPort=45213, infoSecurePort=0, ipcPort=38149, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,528 INFO [Listener at localhost/38149] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33499 2023-07-30 23:55:33,538 WARN [Listener at localhost/42375] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-30 23:55:33,646 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xce9a106233aac7cb: Processing first storage report for DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9 from datanode 19c097c5-9cea-4fb8-ba77-caf607a10197 2023-07-30 23:55:33,647 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xce9a106233aac7cb: from storage DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9 node DatanodeRegistration(127.0.0.1:35861, datanodeUuid=19c097c5-9cea-4fb8-ba77-caf607a10197, infoPort=43231, infoSecurePort=0, ipcPort=42375, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,647 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xce9a106233aac7cb: Processing first storage report for DS-6f9a01aa-2af1-46cd-90fd-1cfc96c39832 from datanode 19c097c5-9cea-4fb8-ba77-caf607a10197 2023-07-30 23:55:33,647 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xce9a106233aac7cb: from storage DS-6f9a01aa-2af1-46cd-90fd-1cfc96c39832 node DatanodeRegistration(127.0.0.1:35861, datanodeUuid=19c097c5-9cea-4fb8-ba77-caf607a10197, infoPort=43231, infoSecurePort=0, ipcPort=42375, storageInfo=lv=-57;cid=testClusterID;nsid=1994887103;c=1690761332596), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-30 23:55:33,650 DEBUG [Listener at localhost/42375] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192 2023-07-30 23:55:33,650 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,651 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,686 INFO [Listener at localhost/42375] util.FSUtils(471): Created version file at hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc with version=8 2023-07-30 23:55:33,686 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/hbase-staging 2023-07-30 23:55:33,688 INFO [Listener at localhost/42375] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:33,688 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,688 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,688 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:33,689 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,689 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:33,689 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:33,690 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40341 2023-07-30 23:55:33,691 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,692 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,693 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40341 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:33,697 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:403410x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:33,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40341-0x101b8a8e9d90007 connected 2023-07-30 23:55:33,709 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:33,710 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:33,710 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:33,711 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40341 2023-07-30 23:55:33,711 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40341 2023-07-30 23:55:33,711 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40341 2023-07-30 23:55:33,712 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40341 2023-07-30 23:55:33,712 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40341 2023-07-30 23:55:33,712 INFO [Listener at localhost/42375] master.HMaster(444): hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc, hbase.cluster.distributed=false 2023-07-30 23:55:33,726 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:33,726 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,726 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,726 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:33,726 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,727 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:33,727 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:33,728 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35865 2023-07-30 23:55:33,728 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:33,730 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:33,730 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,731 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,732 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35865 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:33,737 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:358650x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:33,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35865-0x101b8a8e9d90008 connected 2023-07-30 23:55:33,738 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:33,739 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:33,739 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:33,740 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35865 2023-07-30 23:55:33,740 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35865 2023-07-30 23:55:33,740 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35865 2023-07-30 23:55:33,741 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35865 2023-07-30 23:55:33,741 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35865 2023-07-30 23:55:33,752 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:33,752 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,753 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,753 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:33,753 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,753 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:33,753 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:33,757 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37453 2023-07-30 23:55:33,757 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:33,758 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:33,759 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,760 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,761 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37453 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:33,764 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:374530x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:33,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37453-0x101b8a8e9d90009 connected 2023-07-30 23:55:33,766 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:33,766 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:33,767 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:33,767 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37453 2023-07-30 23:55:33,768 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37453 2023-07-30 23:55:33,768 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37453 2023-07-30 23:55:33,768 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37453 2023-07-30 23:55:33,768 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37453 2023-07-30 23:55:33,782 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:33,782 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,782 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,782 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:33,782 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,783 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:33,783 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:33,787 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43687 2023-07-30 23:55:33,788 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:33,790 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:33,790 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,792 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,794 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43687 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:33,797 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:436870x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:33,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43687-0x101b8a8e9d9000a connected 2023-07-30 23:55:33,799 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:33,799 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:33,800 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:33,803 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43687 2023-07-30 23:55:33,806 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43687 2023-07-30 23:55:33,807 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43687 2023-07-30 23:55:33,807 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43687 2023-07-30 23:55:33,807 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43687 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:33,819 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:33,821 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43645 2023-07-30 23:55:33,821 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:33,822 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:33,823 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,824 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,825 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43645 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:33,828 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:436450x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:33,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43645-0x101b8a8e9d9000b connected 2023-07-30 23:55:33,830 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:33,830 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:33,831 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:33,832 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43645 2023-07-30 23:55:33,832 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43645 2023-07-30 23:55:33,833 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43645 2023-07-30 23:55:33,834 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43645 2023-07-30 23:55:33,834 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43645 2023-07-30 23:55:33,837 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:33,842 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-30 23:55:33,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:33,844 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:33,844 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:33,844 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:33,844 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:33,845 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:33,845 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:33,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-30 23:55:33,847 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,40341,1690761333687 from backup master directory 2023-07-30 23:55:33,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-30 23:55:33,850 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:33,850 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:33,850 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-30 23:55:33,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:33,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/hbase.id with ID: 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:33,901 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:33,905 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:33,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5195ded7 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:33,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5266409b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:33,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:33,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-30 23:55:33,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:33,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:33,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store-tmp 2023-07-30 23:55:33,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:33,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:55:33,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:33,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:33,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:55:33,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:33,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:33,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:33,956 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:33,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:33,960 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/oldWALs, maxLogs=10 2023-07-30 23:55:33,977 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:33,980 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:33,980 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:33,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 2023-07-30 23:55:33,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:33,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:33,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:33,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:33,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:33,990 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:33,992 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-30 23:55:33,993 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-30 23:55:33,994 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:33,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:33,997 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:34,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:34,003 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:34,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59114367, jitterRate=-0.1191272884607315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:55:34,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:34,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-30 23:55:34,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-30 23:55:34,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-30 23:55:34,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-30 23:55:34,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-30 23:55:34,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-30 23:55:34,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-30 23:55:34,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-30 23:55:34,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-30 23:55:34,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-30 23:55:34,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-30 23:55:34,023 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-30 23:55:34,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-30 23:55:34,024 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-30 23:55:34,026 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:34,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-30 23:55:34,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-30 23:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-30 23:55:34,029 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:34,029 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:34,029 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:34,030 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:34,029 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:34,029 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:34,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40341,1690761333687, sessionid=0x101b8a8e9d90007, setting cluster-up flag (Was=false) 2023-07-30 23:55:34,035 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:34,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-30 23:55:34,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:34,049 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:34,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-30 23:55:34,057 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:34,058 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.hbase-snapshot/.tmp 2023-07-30 23:55:34,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:34,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:34,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-30 23:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690761364063 2023-07-30 23:55:34,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-30 23:55:34,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-30 23:55:34,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-30 23:55:34,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-30 23:55:34,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-30 23:55:34,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-30 23:55:34,065 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-30 23:55:34,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-30 23:55:34,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-30 23:55:34,066 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:34,066 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-30 23:55:34,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-30 23:55:34,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-30 23:55:34,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761334066,5,FailOnTimeoutGroup] 2023-07-30 23:55:34,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761334066,5,FailOnTimeoutGroup] 2023-07-30 23:55:34,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-30 23:55:34,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,068 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:34,093 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:34,093 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:34,094 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:55:34,111 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:34,113 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:55:34,115 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:55:34,115 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:55:34,116 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:34,116 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:55:34,118 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:34,118 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:55:34,119 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:34,119 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:55:34,120 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:55:34,121 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:55:34,121 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:34,123 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:55:34,125 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:55:34,128 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:55:34,130 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:55:34,136 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:34,137 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(951): ClusterId : 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:34,137 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(951): ClusterId : 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:34,141 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:34,141 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59195072, jitterRate=-0.11792469024658203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:55:34,142 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:34,142 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:55:34,143 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:55:34,143 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:55:34,143 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:55:34,143 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:55:34,143 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:55:34,143 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:55:34,144 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:55:34,145 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(951): ClusterId : 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:34,145 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(951): ClusterId : 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:34,147 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:34,148 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:34,148 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:34,148 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-30 23:55:34,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-30 23:55:34,149 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:34,149 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:34,152 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-30 23:55:34,153 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:34,153 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:34,153 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:34,153 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:34,154 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-30 23:55:34,155 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:34,155 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:34,155 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:34,157 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:34,157 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ReadOnlyZKClient(139): Connect 0x55cb9924 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:34,158 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ReadOnlyZKClient(139): Connect 0x50e1abb5 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:34,159 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:34,164 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ReadOnlyZKClient(139): Connect 0x72ad48db to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:34,164 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:34,166 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ReadOnlyZKClient(139): Connect 0x6c78459e to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:34,172 DEBUG [RS:0;jenkins-hbase4:35865] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b0c7ded, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:34,172 DEBUG [RS:0;jenkins-hbase4:35865] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f33f254, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:34,173 DEBUG [RS:2;jenkins-hbase4:43687] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7692c256, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:34,173 DEBUG [RS:2;jenkins-hbase4:43687] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@339042e0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:34,178 DEBUG [RS:3;jenkins-hbase4:43645] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@66802eee, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:34,178 DEBUG [RS:3;jenkins-hbase4:43645] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2382bc5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:34,178 DEBUG [RS:1;jenkins-hbase4:37453] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e264fd9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:34,179 DEBUG [RS:1;jenkins-hbase4:37453] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5a19c0c5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:34,183 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:35865 2023-07-30 23:55:34,183 INFO [RS:0;jenkins-hbase4:35865] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:34,183 INFO [RS:0;jenkins-hbase4:35865] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:34,183 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:34,184 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40341,1690761333687 with isa=jenkins-hbase4.apache.org/172.31.14.131:35865, startcode=1690761333726 2023-07-30 23:55:34,184 DEBUG [RS:0;jenkins-hbase4:35865] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:34,186 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:43687 2023-07-30 23:55:34,186 INFO [RS:2;jenkins-hbase4:43687] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:34,186 INFO [RS:2;jenkins-hbase4:43687] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:34,188 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:34,186 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:43645 2023-07-30 23:55:34,189 INFO [RS:3;jenkins-hbase4:43645] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:34,189 INFO [RS:3;jenkins-hbase4:43645] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:34,190 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:34,190 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45335, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:34,190 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40341,1690761333687 with isa=jenkins-hbase4.apache.org/172.31.14.131:43645, startcode=1690761333818 2023-07-30 23:55:34,191 DEBUG [RS:3;jenkins-hbase4:43645] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:34,191 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40341,1690761333687 with isa=jenkins-hbase4.apache.org/172.31.14.131:43687, startcode=1690761333781 2023-07-30 23:55:34,191 DEBUG [RS:2;jenkins-hbase4:43687] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:34,196 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:37453 2023-07-30 23:55:34,199 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40341] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,199 INFO [RS:1;jenkins-hbase4:37453] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:34,199 INFO [RS:1;jenkins-hbase4:37453] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:34,199 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:34,200 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54137, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:34,200 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40341] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,201 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:55:34,201 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35081 2023-07-30 23:55:34,201 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:34,202 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:55:34,202 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35081 2023-07-30 23:55:34,202 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:34,204 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40869, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:34,204 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40341] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,204 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40341,1690761333687 with isa=jenkins-hbase4.apache.org/172.31.14.131:37453, startcode=1690761333752 2023-07-30 23:55:34,204 DEBUG [RS:1;jenkins-hbase4:37453] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:34,205 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:55:34,205 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35081 2023-07-30 23:55:34,205 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:34,209 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55587, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:34,209 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40341] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,210 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:55:34,210 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35081 2023-07-30 23:55:34,210 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:34,213 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:34,215 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ZKUtil(162): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,215 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ZKUtil(162): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,215 WARN [RS:2;jenkins-hbase4:43687] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:34,215 WARN [RS:3;jenkins-hbase4:43645] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:34,215 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ZKUtil(162): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,215 INFO [RS:3;jenkins-hbase4:43645] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:34,215 INFO [RS:2;jenkins-hbase4:43687] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:34,215 INFO [RS:3;jenkins-hbase4:43645] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:34,215 WARN [RS:0;jenkins-hbase4:35865] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:34,216 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,216 INFO [RS:2;jenkins-hbase4:43687] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:34,216 INFO [RS:0;jenkins-hbase4:35865] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:34,216 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,216 INFO [RS:0;jenkins-hbase4:35865] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:34,216 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,217 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43645,1690761333818] 2023-07-30 23:55:34,217 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37453,1690761333752] 2023-07-30 23:55:34,217 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35865,1690761333726] 2023-07-30 23:55:34,217 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43687,1690761333781] 2023-07-30 23:55:34,217 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ZKUtil(162): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,218 WARN [RS:1;jenkins-hbase4:37453] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:34,218 INFO [RS:1;jenkins-hbase4:37453] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:34,219 INFO [RS:1;jenkins-hbase4:37453] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:34,219 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,230 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ZKUtil(162): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,230 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ZKUtil(162): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,230 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ZKUtil(162): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,231 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ZKUtil(162): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,231 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ZKUtil(162): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,232 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ZKUtil(162): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,232 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ZKUtil(162): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,233 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ZKUtil(162): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,234 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:34,234 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:34,234 INFO [RS:2;jenkins-hbase4:43687] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:34,234 INFO [RS:0;jenkins-hbase4:35865] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:34,237 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ZKUtil(162): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,237 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ZKUtil(162): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,238 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ZKUtil(162): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,238 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ZKUtil(162): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,238 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ZKUtil(162): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,239 INFO [RS:2;jenkins-hbase4:43687] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:34,239 INFO [RS:2;jenkins-hbase4:43687] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:34,239 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,240 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ZKUtil(162): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,240 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ZKUtil(162): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,240 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ZKUtil(162): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,242 DEBUG [RS:1;jenkins-hbase4:37453] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:34,242 DEBUG [RS:3;jenkins-hbase4:43645] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:34,242 INFO [RS:1;jenkins-hbase4:37453] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:34,242 INFO [RS:3;jenkins-hbase4:43645] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:34,242 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:34,243 INFO [RS:0;jenkins-hbase4:35865] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:34,247 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,247 INFO [RS:0;jenkins-hbase4:35865] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,247 DEBUG [RS:2;jenkins-hbase4:43687] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,252 INFO [RS:3;jenkins-hbase4:43645] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:34,252 INFO [RS:1;jenkins-hbase4:37453] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:34,252 INFO [RS:3;jenkins-hbase4:43645] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:34,252 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,252 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:34,255 INFO [RS:1;jenkins-hbase4:37453] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:34,255 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,257 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,257 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:34,258 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,258 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:34,259 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,259 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,260 DEBUG [RS:0;jenkins-hbase4:35865] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,262 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,262 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,263 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,263 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,264 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,264 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,264 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,264 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,264 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,264 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,265 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,264 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:34,265 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:1;jenkins-hbase4:37453] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,265 DEBUG [RS:3;jenkins-hbase4:43645] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:34,270 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,271 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,271 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,275 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,275 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,275 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,285 INFO [RS:2;jenkins-hbase4:43687] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:34,285 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43687,1690761333781-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,288 INFO [RS:0;jenkins-hbase4:35865] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:34,288 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35865,1690761333726-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,290 INFO [RS:3;jenkins-hbase4:43645] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:34,290 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43645,1690761333818-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,293 INFO [RS:1;jenkins-hbase4:37453] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:34,293 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37453,1690761333752-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,302 INFO [RS:2;jenkins-hbase4:43687] regionserver.Replication(203): jenkins-hbase4.apache.org,43687,1690761333781 started 2023-07-30 23:55:34,302 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43687,1690761333781, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43687, sessionid=0x101b8a8e9d9000a 2023-07-30 23:55:34,302 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:34,302 DEBUG [RS:2;jenkins-hbase4:43687] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,302 DEBUG [RS:2;jenkins-hbase4:43687] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43687,1690761333781' 2023-07-30 23:55:34,302 DEBUG [RS:2;jenkins-hbase4:43687] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-30 23:55:34,303 DEBUG [RS:2;jenkins-hbase4:43687] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-30 23:55:34,305 DEBUG [jenkins-hbase4:40341] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-30 23:55:34,305 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:34,305 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:34,305 DEBUG [jenkins-hbase4:40341] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:34,305 DEBUG [RS:2;jenkins-hbase4:43687] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,305 DEBUG [RS:2;jenkins-hbase4:43687] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43687,1690761333781' 2023-07-30 23:55:34,305 DEBUG [RS:2;jenkins-hbase4:43687] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-30 23:55:34,306 DEBUG [RS:2;jenkins-hbase4:43687] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-30 23:55:34,308 INFO [RS:0;jenkins-hbase4:35865] regionserver.Replication(203): jenkins-hbase4.apache.org,35865,1690761333726 started 2023-07-30 23:55:34,308 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,35865,1690761333726, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35865, sessionid=0x101b8a8e9d90008 2023-07-30 23:55:34,308 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:34,308 DEBUG [RS:0;jenkins-hbase4:35865] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,308 DEBUG [RS:0;jenkins-hbase4:35865] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35865,1690761333726' 2023-07-30 23:55:34,308 DEBUG [RS:0;jenkins-hbase4:35865] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-30 23:55:34,309 INFO [RS:3;jenkins-hbase4:43645] regionserver.Replication(203): jenkins-hbase4.apache.org,43645,1690761333818 started 2023-07-30 23:55:34,309 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43645,1690761333818, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43645, sessionid=0x101b8a8e9d9000b 2023-07-30 23:55:34,309 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:34,309 DEBUG [RS:3;jenkins-hbase4:43645] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,310 DEBUG [RS:3;jenkins-hbase4:43645] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43645,1690761333818' 2023-07-30 23:55:34,310 DEBUG [RS:3;jenkins-hbase4:43645] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-30 23:55:34,310 DEBUG [RS:2;jenkins-hbase4:43687] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:34,310 INFO [RS:2;jenkins-hbase4:43687] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:34,310 INFO [RS:2;jenkins-hbase4:43687] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:34,310 DEBUG [RS:3;jenkins-hbase4:43645] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-30 23:55:34,311 DEBUG [jenkins-hbase4:40341] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:34,311 DEBUG [jenkins-hbase4:40341] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:34,311 DEBUG [jenkins-hbase4:40341] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-30 23:55:34,311 DEBUG [jenkins-hbase4:40341] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-30 23:55:34,312 DEBUG [jenkins-hbase4:40341] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:34,310 DEBUG [RS:0;jenkins-hbase4:35865] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-30 23:55:34,318 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43687,1690761333781, state=OPENING 2023-07-30 23:55:34,323 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:34,323 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:34,323 DEBUG [RS:0;jenkins-hbase4:35865] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:34,323 DEBUG [RS:0;jenkins-hbase4:35865] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35865,1690761333726' 2023-07-30 23:55:34,323 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:34,323 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:34,323 DEBUG [RS:0;jenkins-hbase4:35865] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-30 23:55:34,323 DEBUG [RS:3;jenkins-hbase4:43645] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:34,323 DEBUG [RS:3;jenkins-hbase4:43645] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43645,1690761333818' 2023-07-30 23:55:34,323 DEBUG [RS:3;jenkins-hbase4:43645] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-30 23:55:34,324 DEBUG [RS:0;jenkins-hbase4:35865] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-30 23:55:34,324 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-30 23:55:34,324 DEBUG [RS:0;jenkins-hbase4:35865] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:34,325 DEBUG [RS:3;jenkins-hbase4:43645] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-30 23:55:34,325 INFO [RS:0;jenkins-hbase4:35865] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:34,325 INFO [RS:0;jenkins-hbase4:35865] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:34,326 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:34,326 DEBUG [RS:3;jenkins-hbase4:43645] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:34,326 INFO [RS:3;jenkins-hbase4:43645] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:34,326 INFO [RS:3;jenkins-hbase4:43645] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:34,327 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-30 23:55:34,327 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43687,1690761333781}] 2023-07-30 23:55:34,336 INFO [RS:1;jenkins-hbase4:37453] regionserver.Replication(203): jenkins-hbase4.apache.org,37453,1690761333752 started 2023-07-30 23:55:34,336 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37453,1690761333752, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37453, sessionid=0x101b8a8e9d90009 2023-07-30 23:55:34,338 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:34,338 DEBUG [RS:1;jenkins-hbase4:37453] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,338 DEBUG [RS:1;jenkins-hbase4:37453] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37453,1690761333752' 2023-07-30 23:55:34,338 DEBUG [RS:1;jenkins-hbase4:37453] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-30 23:55:34,338 DEBUG [RS:1;jenkins-hbase4:37453] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-30 23:55:34,340 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:34,340 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:34,340 DEBUG [RS:1;jenkins-hbase4:37453] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:34,340 DEBUG [RS:1;jenkins-hbase4:37453] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37453,1690761333752' 2023-07-30 23:55:34,340 DEBUG [RS:1;jenkins-hbase4:37453] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-30 23:55:34,341 DEBUG [RS:1;jenkins-hbase4:37453] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-30 23:55:34,341 DEBUG [RS:1;jenkins-hbase4:37453] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:34,342 INFO [RS:1;jenkins-hbase4:37453] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:34,342 INFO [RS:1;jenkins-hbase4:37453] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:34,410 INFO [RS:2;jenkins-hbase4:43687] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:34,413 INFO [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43687%2C1690761333781.jenkins-hbase4.apache.org%2C43687%2C1690761333781.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:34,425 INFO [RS:0;jenkins-hbase4:35865] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:34,428 INFO [RS:3;jenkins-hbase4:43645] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:34,430 INFO [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35865%2C1690761333726.jenkins-hbase4.apache.org%2C35865%2C1690761333726.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:34,434 INFO [RS:3;jenkins-hbase4:43645] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43645%2C1690761333818.jenkins-hbase4.apache.org%2C43645%2C1690761333818.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:34,440 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:34,441 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:34,441 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:34,442 INFO [RS:1;jenkins-hbase4:37453] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:34,445 INFO [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781/jenkins-hbase4.apache.org%2C43687%2C1690761333781.jenkins-hbase4.apache.org%2C43687%2C1690761333781.regiongroup-0.1690761334415 2023-07-30 23:55:34,446 INFO [RS:1;jenkins-hbase4:37453] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37453%2C1690761333752.jenkins-hbase4.apache.org%2C37453%2C1690761333752.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:34,446 DEBUG [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK]] 2023-07-30 23:55:34,463 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:34,464 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:34,466 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:34,468 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:34,472 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:34,472 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:34,479 INFO [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726/jenkins-hbase4.apache.org%2C35865%2C1690761333726.jenkins-hbase4.apache.org%2C35865%2C1690761333726.regiongroup-0.1690761334433 2023-07-30 23:55:34,479 DEBUG [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK]] 2023-07-30 23:55:34,480 INFO [RS:3;jenkins-hbase4:43645] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818/jenkins-hbase4.apache.org%2C43645%2C1690761333818.jenkins-hbase4.apache.org%2C43645%2C1690761333818.regiongroup-0.1690761334435 2023-07-30 23:55:34,481 DEBUG [RS:3;jenkins-hbase4:43645] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK]] 2023-07-30 23:55:34,493 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,495 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:34,494 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:34,494 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:34,496 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:34,501 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33480, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:34,507 INFO [RS:1;jenkins-hbase4:37453] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752/jenkins-hbase4.apache.org%2C37453%2C1690761333752.jenkins-hbase4.apache.org%2C37453%2C1690761333752.regiongroup-0.1690761334447 2023-07-30 23:55:34,507 DEBUG [RS:1;jenkins-hbase4:37453] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:34,512 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-30 23:55:34,512 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:34,512 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:34,512 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:34,515 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43687%2C1690761333781.meta, suffix=.meta, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:34,537 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:34,537 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:34,537 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:34,547 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781/jenkins-hbase4.apache.org%2C43687%2C1690761333781.meta.1690761334515.meta 2023-07-30 23:55:34,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:34,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:34,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-30 23:55:34,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-30 23:55:34,548 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-30 23:55:34,548 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-30 23:55:34,549 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:34,549 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-30 23:55:34,549 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-30 23:55:34,553 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:55:34,554 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:55:34,554 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:55:34,555 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:55:34,556 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:34,556 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:55:34,558 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:34,558 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:34,558 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:55:34,559 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:34,559 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:55:34,561 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:55:34,561 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:55:34,561 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:55:34,562 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:34,564 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:55:34,567 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:55:34,572 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:55:34,574 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:55:34,576 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59742690, jitterRate=-0.10976454615592957}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:55:34,576 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:55:34,577 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690761334493 2023-07-30 23:55:34,583 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-30 23:55:34,584 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-30 23:55:34,585 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43687,1690761333781, state=OPEN 2023-07-30 23:55:34,589 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-30 23:55:34,589 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-30 23:55:34,592 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-30 23:55:34,592 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43687,1690761333781 in 263 msec 2023-07-30 23:55:34,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-30 23:55:34,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 444 msec 2023-07-30 23:55:34,599 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 537 msec 2023-07-30 23:55:34,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690761334599, completionTime=-1 2023-07-30 23:55:34,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-30 23:55:34,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-30 23:55:34,603 DEBUG [hconnection-0x1dbe2a9c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:34,605 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33482, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:34,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-30 23:55:34,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690761394607 2023-07-30 23:55:34,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690761454607 2023-07-30 23:55:34,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 8 msec 2023-07-30 23:55:34,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40341,1690761333687-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40341,1690761333687-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40341,1690761333687-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40341, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:34,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-30 23:55:34,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:34,625 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-30 23:55:34,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-30 23:55:34,627 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-30 23:55:34,629 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-30 23:55:34,634 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:55:34,635 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/hbase/namespace/51a075540234ea8884d31403732b8710 empty. 2023-07-30 23:55:34,636 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:55:34,636 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-30 23:55:34,708 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:34,710 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 51a075540234ea8884d31403732b8710, NAME => 'hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp 2023-07-30 23:55:34,732 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:34,733 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 51a075540234ea8884d31403732b8710, disabling compactions & flushes 2023-07-30 23:55:34,733 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:34,733 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:34,733 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. after waiting 0 ms 2023-07-30 23:55:34,733 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:34,733 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:34,733 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:55:34,737 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-30 23:55:34,740 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761334739"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690761334739"}]},"ts":"1690761334739"} 2023-07-30 23:55:34,744 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-30 23:55:34,745 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-30 23:55:34,746 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761334745"}]},"ts":"1690761334745"} 2023-07-30 23:55:34,748 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-30 23:55:34,753 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:34,754 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:34,754 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:34,754 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-30 23:55:34,754 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-30 23:55:34,754 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:34,754 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN}] 2023-07-30 23:55:34,758 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN 2023-07-30 23:55:34,759 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43687,1690761333781; forceNewPlan=false, retain=false 2023-07-30 23:55:34,909 INFO [jenkins-hbase4:40341] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-30 23:55:34,911 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=51a075540234ea8884d31403732b8710, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:34,911 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761334911"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761334911"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761334911"}]},"ts":"1690761334911"} 2023-07-30 23:55:34,913 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 51a075540234ea8884d31403732b8710, server=jenkins-hbase4.apache.org,43687,1690761333781}] 2023-07-30 23:55:35,073 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:35,073 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:35,076 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43687%2C1690761333781.jenkins-hbase4.apache.org%2C43687%2C1690761333781.regiongroup-1, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:35,091 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:35,093 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:35,093 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:35,097 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781/jenkins-hbase4.apache.org%2C43687%2C1690761333781.jenkins-hbase4.apache.org%2C43687%2C1690761333781.regiongroup-1.1690761335076 2023-07-30 23:55:35,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK]] 2023-07-30 23:55:35,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 51a075540234ea8884d31403732b8710, NAME => 'hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:35,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:35,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,100 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,102 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info 2023-07-30 23:55:35,102 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info 2023-07-30 23:55:35,104 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51a075540234ea8884d31403732b8710 columnFamilyName info 2023-07-30 23:55:35,105 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(310): Store=51a075540234ea8884d31403732b8710/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:35,107 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,108 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,111 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:35,113 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:35,114 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 51a075540234ea8884d31403732b8710; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64302345, jitterRate=-0.04182039201259613}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:55:35,114 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:55:35,115 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., pid=6, masterSystemTime=1690761335066 2023-07-30 23:55:35,118 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:35,119 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:35,119 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=51a075540234ea8884d31403732b8710, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:35,120 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761335119"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761335119"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761335119"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761335119"}]},"ts":"1690761335119"} 2023-07-30 23:55:35,125 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-30 23:55:35,125 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 51a075540234ea8884d31403732b8710, server=jenkins-hbase4.apache.org,43687,1690761333781 in 209 msec 2023-07-30 23:55:35,128 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-30 23:55:35,129 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN in 371 msec 2023-07-30 23:55:35,130 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-30 23:55:35,130 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761335130"}]},"ts":"1690761335130"} 2023-07-30 23:55:35,133 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-30 23:55:35,136 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-30 23:55:35,138 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 513 msec 2023-07-30 23:55:35,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-30 23:55:35,231 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-30 23:55:35,231 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:35,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-30 23:55:35,250 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-30 23:55:35,255 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 18 msec 2023-07-30 23:55:35,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-30 23:55:35,269 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-30 23:55:35,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 12 msec 2023-07-30 23:55:35,285 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-30 23:55:35,288 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-30 23:55:35,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.438sec 2023-07-30 23:55:35,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-30 23:55:35,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-30 23:55:35,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-30 23:55:35,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40341,1690761333687-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-30 23:55:35,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40341,1690761333687-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-30 23:55:35,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-30 23:55:35,342 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x4d6e064e to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:35,351 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5eca9e3a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:35,353 DEBUG [hconnection-0x4f3eec21-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:35,355 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33488, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:35,357 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:35,368 DEBUG [Listener at localhost/42375] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-30 23:55:35,370 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47008, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-30 23:55:35,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:35,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-30 23:55:35,390 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-30 23:55:35,392 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-30 23:55:35,394 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,395 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff empty. 2023-07-30 23:55:35,395 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-30 23:55:35,395 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-30 23:55:35,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:35,421 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:35,422 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 409ccc1344e052df313f0ebb0776f6ff, NAME => 't1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp 2023-07-30 23:55:35,469 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:35,470 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 409ccc1344e052df313f0ebb0776f6ff, disabling compactions & flushes 2023-07-30 23:55:35,470 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,470 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,470 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. after waiting 0 ms 2023-07-30 23:55:35,470 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,470 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,470 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 409ccc1344e052df313f0ebb0776f6ff: 2023-07-30 23:55:35,475 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-30 23:55:35,477 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761335477"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690761335477"}]},"ts":"1690761335477"} 2023-07-30 23:55:35,480 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-30 23:55:35,481 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-30 23:55:35,481 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761335481"}]},"ts":"1690761335481"} 2023-07-30 23:55:35,483 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-30 23:55:35,487 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:35,488 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:35,488 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:35,488 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:35,489 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN}] 2023-07-30 23:55:35,491 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN 2023-07-30 23:55:35,492 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36237,1690761329770; forceNewPlan=false, retain=false 2023-07-30 23:55:35,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:35,642 INFO [jenkins-hbase4:43527] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-30 23:55:35,644 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=409ccc1344e052df313f0ebb0776f6ff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:35,645 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761335644"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761335644"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761335644"}]},"ts":"1690761335644"} 2023-07-30 23:55:35,648 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 409ccc1344e052df313f0ebb0776f6ff, server=jenkins-hbase4.apache.org,36237,1690761329770}] 2023-07-30 23:55:35,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:35,802 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:35,802 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:35,804 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46340, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:35,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:35,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:55:35,833 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:55:35,834 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:55:35,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817 2023-07-30 23:55:35,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:55:35,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 409ccc1344e052df313f0ebb0776f6ff, NAME => 't1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:35,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:35,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,841 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,843 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1 2023-07-30 23:55:35,843 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1 2023-07-30 23:55:35,844 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 409ccc1344e052df313f0ebb0776f6ff columnFamilyName cf1 2023-07-30 23:55:35,844 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(310): Store=409ccc1344e052df313f0ebb0776f6ff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:35,844 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,846 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep 2023-07-30 23:55:35,846 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep 2023-07-30 23:55:35,847 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 409ccc1344e052df313f0ebb0776f6ff columnFamilyName norep 2023-07-30 23:55:35,847 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(310): Store=409ccc1344e052df313f0ebb0776f6ff/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:35,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:55:35,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:35,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:35,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 409ccc1344e052df313f0ebb0776f6ff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60846713, jitterRate=-0.0933133214712143}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:55:35,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 409ccc1344e052df313f0ebb0776f6ff: 2023-07-30 23:55:35,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff., pid=11, masterSystemTime=1690761335802 2023-07-30 23:55:35,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:35,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=409ccc1344e052df313f0ebb0776f6ff, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:35,872 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761335871"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761335871"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761335871"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761335871"}]},"ts":"1690761335871"} 2023-07-30 23:55:35,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-30 23:55:35,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 409ccc1344e052df313f0ebb0776f6ff, server=jenkins-hbase4.apache.org,36237,1690761329770 in 227 msec 2023-07-30 23:55:35,884 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-30 23:55:35,885 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN in 391 msec 2023-07-30 23:55:35,886 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-30 23:55:35,886 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761335886"}]},"ts":"1690761335886"} 2023-07-30 23:55:35,889 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-30 23:55:35,892 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-30 23:55:35,894 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 507 msec 2023-07-30 23:55:36,013 INFO [jenkins-hbase4:42313Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:36,013 INFO [jenkins-hbase4:36237Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:36,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:36,015 INFO [Listener at localhost/42375] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-30 23:55:36,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:36,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-30 23:55:36,022 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-30 23:55:36,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-30 23:55:36,025 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-30 23:55:36,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:36,027 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t2_syncup/bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,028 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t2_syncup/bffda5817d519b05008d136099bf920f empty. 2023-07-30 23:55:36,029 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t2_syncup/bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,029 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-30 23:55:36,069 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:36,071 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => bffda5817d519b05008d136099bf920f, NAME => 't2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.tmp 2023-07-30 23:55:36,098 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:36,098 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing bffda5817d519b05008d136099bf920f, disabling compactions & flushes 2023-07-30 23:55:36,098 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,098 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,098 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. after waiting 0 ms 2023-07-30 23:55:36,098 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,098 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,098 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for bffda5817d519b05008d136099bf920f: 2023-07-30 23:55:36,103 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-30 23:55:36,105 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761336104"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690761336104"}]},"ts":"1690761336104"} 2023-07-30 23:55:36,107 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-30 23:55:36,108 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-30 23:55:36,108 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761336108"}]},"ts":"1690761336108"} 2023-07-30 23:55:36,110 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-30 23:55:36,116 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:36,118 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:36,118 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:36,118 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:36,118 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN}] 2023-07-30 23:55:36,121 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN 2023-07-30 23:55:36,122 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42313,1690761329724; forceNewPlan=false, retain=false 2023-07-30 23:55:36,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:36,273 INFO [jenkins-hbase4:43527] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-30 23:55:36,274 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=bffda5817d519b05008d136099bf920f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:36,275 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761336274"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761336274"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761336274"}]},"ts":"1690761336274"} 2023-07-30 23:55:36,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure bffda5817d519b05008d136099bf920f, server=jenkins-hbase4.apache.org,42313,1690761329724}] 2023-07-30 23:55:36,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:36,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bffda5817d519b05008d136099bf920f, NAME => 't2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:36,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:36,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,438 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,440 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1 2023-07-30 23:55:36,440 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1 2023-07-30 23:55:36,440 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bffda5817d519b05008d136099bf920f columnFamilyName cf1 2023-07-30 23:55:36,441 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(310): Store=bffda5817d519b05008d136099bf920f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:36,441 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,442 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep 2023-07-30 23:55:36,443 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep 2023-07-30 23:55:36,443 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bffda5817d519b05008d136099bf920f columnFamilyName norep 2023-07-30 23:55:36,444 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(310): Store=bffda5817d519b05008d136099bf920f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:36,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:55:36,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bffda5817d519b05008d136099bf920f 2023-07-30 23:55:36,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:36,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bffda5817d519b05008d136099bf920f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62344876, jitterRate=-0.07098895311355591}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:55:36,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bffda5817d519b05008d136099bf920f: 2023-07-30 23:55:36,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f., pid=14, masterSystemTime=1690761336430 2023-07-30 23:55:36,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:36,457 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=bffda5817d519b05008d136099bf920f, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:36,457 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761336457"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761336457"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761336457"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761336457"}]},"ts":"1690761336457"} 2023-07-30 23:55:36,462 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-30 23:55:36,462 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure bffda5817d519b05008d136099bf920f, server=jenkins-hbase4.apache.org,42313,1690761329724 in 182 msec 2023-07-30 23:55:36,464 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-30 23:55:36,466 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN in 344 msec 2023-07-30 23:55:36,466 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-30 23:55:36,467 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761336466"}]},"ts":"1690761336466"} 2023-07-30 23:55:36,483 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-30 23:55:36,485 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-30 23:55:36,487 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 468 msec 2023-07-30 23:55:36,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:36,630 INFO [Listener at localhost/42375] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-30 23:55:36,633 DEBUG [Listener at localhost/42375] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-30 23:55:36,635 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-30 23:55:36,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:36,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-30 23:55:36,642 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-30 23:55:36,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-30 23:55:36,648 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-30 23:55:36,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:36,650 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:36,651 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 empty. 2023-07-30 23:55:36,652 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:36,652 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-30 23:55:36,681 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:36,682 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6494f975164328a819a48f1beabe91f9, NAME => 't1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp 2023-07-30 23:55:36,696 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:36,696 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 6494f975164328a819a48f1beabe91f9, disabling compactions & flushes 2023-07-30 23:55:36,696 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:36,696 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:36,696 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. after waiting 0 ms 2023-07-30 23:55:36,696 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:36,696 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:36,696 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:55:36,702 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-30 23:55:36,703 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761336702"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690761336702"}]},"ts":"1690761336702"} 2023-07-30 23:55:36,705 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-30 23:55:36,706 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-30 23:55:36,706 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761336706"}]},"ts":"1690761336706"} 2023-07-30 23:55:36,708 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-30 23:55:36,713 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:36,714 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:36,714 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:36,714 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-30 23:55:36,714 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-30 23:55:36,714 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:36,714 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN}] 2023-07-30 23:55:36,716 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN 2023-07-30 23:55:36,717 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35865,1690761333726; forceNewPlan=false, retain=false 2023-07-30 23:55:36,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:36,868 INFO [jenkins-hbase4:40341] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-30 23:55:36,869 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6494f975164328a819a48f1beabe91f9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:36,870 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761336869"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761336869"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761336869"}]},"ts":"1690761336869"} 2023-07-30 23:55:36,872 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 6494f975164328a819a48f1beabe91f9, server=jenkins-hbase4.apache.org,35865,1690761333726}] 2023-07-30 23:55:36,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:36,984 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:55:37,025 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:37,025 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:37,028 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48270, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:37,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:37,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:37,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35865%2C1690761333726.jenkins-hbase4.apache.org%2C35865%2C1690761333726.regiongroup-1, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:37,065 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:37,065 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:37,065 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:37,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726/jenkins-hbase4.apache.org%2C35865%2C1690761333726.jenkins-hbase4.apache.org%2C35865%2C1690761333726.regiongroup-1.1690761337040 2023-07-30 23:55:37,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:37,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6494f975164328a819a48f1beabe91f9, NAME => 't1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:37,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:37,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,101 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,108 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1 2023-07-30 23:55:37,113 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1 2023-07-30 23:55:37,113 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6494f975164328a819a48f1beabe91f9 columnFamilyName cf1 2023-07-30 23:55:37,115 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(310): Store=6494f975164328a819a48f1beabe91f9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:37,115 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,119 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/norep 2023-07-30 23:55:37,119 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/norep 2023-07-30 23:55:37,119 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6494f975164328a819a48f1beabe91f9 columnFamilyName norep 2023-07-30 23:55:37,120 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(310): Store=6494f975164328a819a48f1beabe91f9/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:37,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:55:37,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:37,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:37,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6494f975164328a819a48f1beabe91f9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64818394, jitterRate=-0.03413066267967224}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:55:37,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:55:37,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9., pid=11, masterSystemTime=1690761337025 2023-07-30 23:55:37,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:37,153 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6494f975164328a819a48f1beabe91f9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:37,154 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761337153"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761337153"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761337153"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761337153"}]},"ts":"1690761337153"} 2023-07-30 23:55:37,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:37,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-30 23:55:37,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 6494f975164328a819a48f1beabe91f9, server=jenkins-hbase4.apache.org,35865,1690761333726 in 285 msec 2023-07-30 23:55:37,162 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-30 23:55:37,162 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN in 446 msec 2023-07-30 23:55:37,163 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-30 23:55:37,163 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761337163"}]},"ts":"1690761337163"} 2023-07-30 23:55:37,165 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-30 23:55:37,169 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-30 23:55:37,173 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 532 msec 2023-07-30 23:55:37,175 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-30 23:55:37,176 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-30 23:55:37,177 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-30 23:55:37,177 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-30 23:55:37,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-30 23:55:37,253 INFO [Listener at localhost/42375] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-30 23:55:37,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:37,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-30 23:55:37,259 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-30 23:55:37,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-30 23:55:37,260 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-30 23:55:37,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:37,262 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,263 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 empty. 2023-07-30 23:55:37,264 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,265 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-30 23:55:37,295 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-30 23:55:37,297 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => ee738a1d08d169b10fce15cc4dc0d043, NAME => 't2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.tmp 2023-07-30 23:55:37,313 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:37,313 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing ee738a1d08d169b10fce15cc4dc0d043, disabling compactions & flushes 2023-07-30 23:55:37,313 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,313 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,313 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. after waiting 0 ms 2023-07-30 23:55:37,313 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,313 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,314 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:55:37,317 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-30 23:55:37,318 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761337318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690761337318"}]},"ts":"1690761337318"} 2023-07-30 23:55:37,320 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-30 23:55:37,322 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-30 23:55:37,322 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761337322"}]},"ts":"1690761337322"} 2023-07-30 23:55:37,324 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-30 23:55:37,328 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-30 23:55:37,329 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-30 23:55:37,330 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-30 23:55:37,330 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-30 23:55:37,330 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-30 23:55:37,330 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-30 23:55:37,330 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN}] 2023-07-30 23:55:37,332 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN 2023-07-30 23:55:37,333 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43687,1690761333781; forceNewPlan=false, retain=false 2023-07-30 23:55:37,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:37,483 INFO [jenkins-hbase4:40341] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-30 23:55:37,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:37,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761337485"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761337485"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761337485"}]},"ts":"1690761337485"} 2023-07-30 23:55:37,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure ee738a1d08d169b10fce15cc4dc0d043, server=jenkins-hbase4.apache.org,43687,1690761333781}] 2023-07-30 23:55:37,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:37,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee738a1d08d169b10fce15cc4dc0d043, NAME => 't2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:37,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:37,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,649 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,651 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1 2023-07-30 23:55:37,651 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1 2023-07-30 23:55:37,651 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee738a1d08d169b10fce15cc4dc0d043 columnFamilyName cf1 2023-07-30 23:55:37,652 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(310): Store=ee738a1d08d169b10fce15cc4dc0d043/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:37,652 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,654 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/norep 2023-07-30 23:55:37,654 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/norep 2023-07-30 23:55:37,654 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee738a1d08d169b10fce15cc4dc0d043 columnFamilyName norep 2023-07-30 23:55:37,655 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(310): Store=ee738a1d08d169b10fce15cc4dc0d043/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:37,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:55:37,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:37,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-30 23:55:37,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee738a1d08d169b10fce15cc4dc0d043; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71724124, jitterRate=0.06877273321151733}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:55:37,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:55:37,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043., pid=14, masterSystemTime=1690761337642 2023-07-30 23:55:37,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:37,669 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:37,669 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761337669"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761337669"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761337669"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761337669"}]},"ts":"1690761337669"} 2023-07-30 23:55:37,674 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-30 23:55:37,674 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure ee738a1d08d169b10fce15cc4dc0d043, server=jenkins-hbase4.apache.org,43687,1690761333781 in 183 msec 2023-07-30 23:55:37,677 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-30 23:55:37,677 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN in 344 msec 2023-07-30 23:55:37,678 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-30 23:55:37,678 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690761337678"}]},"ts":"1690761337678"} 2023-07-30 23:55:37,680 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-30 23:55:37,682 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-30 23:55:37,685 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 428 msec 2023-07-30 23:55:37,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40341] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-30 23:55:37,865 INFO [Listener at localhost/42375] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-30 23:55:37,866 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x2db404e1 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:37,874 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@652c226a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:37,874 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x76c6a82b to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:37,885 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b58391e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:37,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:52585:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-30 23:55:37,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-30 23:55:37,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-30 23:55:37,932 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-30 23:55:37,932 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47d7b834check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:37,936 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47d7b834check-peer-cluster-id0x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:37,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47d7b834check-peer-cluster-id-0x101b8a8e9d90014 connected 2023-07-30 23:55:38,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-30 23:55:38,040 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@47d7b834check-peer-cluster-id-0x101b8a8e9d90014, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:38,050 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:38,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-30 23:55:38,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-30 23:55:38,232 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42313] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-30 23:55:38,233 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36237] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-30 23:55:38,233 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-30 23:55:38,253 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-30 23:55:38,254 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-30 23:55:38,291 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:55:38,291 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-30 23:55:38,322 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:55:38,326 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-30 23:55:38,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.HMaster(3641): Remote procedure done, pid=17 2023-07-30 23:55:38,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,42313,1690761329724 suceeded 2023-07-30 23:55:38,336 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 271 msec 2023-07-30 23:55:38,340 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-30 23:55:38,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.HMaster(3641): Remote procedure done, pid=16 2023-07-30 23:55:38,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,36237,1690761329770 suceeded 2023-07-30 23:55:38,345 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-07-30 23:55:38,345 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 280 msec 2023-07-30 23:55:38,345 INFO [PEWorker-1] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:52585:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-30 23:55:38,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 442 msec 2023-07-30 23:55:38,363 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x2703aa98 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:38,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@570aef90, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:38,373 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:38,381 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x4d00cd60 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:38,384 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:38,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d90016 connected 2023-07-30 23:55:38,388 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:38,388 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@558ed653, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:38,390 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:38,392 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0 2023-07-30 23:55:38,401 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:38,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d90018 connected 2023-07-30 23:55:38,403 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:38,403 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1 2023-07-30 23:55:38,403 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:38,403 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:38,408 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1 2023-07-30 23:55:38,408 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1 2023-07-30 23:55:38,408 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0 2023-07-30 23:55:38,409 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:38,408 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:38,409 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0 2023-07-30 23:55:38,415 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0 2023-07-30 23:55:38,427 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1 2023-07-30 23:55:38,504 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:38,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-30 23:55:38,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:38,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:38,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-30 23:55:38,526 DEBUG [Listener at localhost/42375] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-30 23:55:38,528 DEBUG [hconnection-0x7764dc29-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,530 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32850, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,537 DEBUG [Listener at localhost/42375] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,538 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46352, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,614 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:38,628 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:38,630 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-30 23:55:38,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6139 2023-07-30 23:55:38,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 55 WAL entries eligible for replication 2023-07-30 23:55:38,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6139 2023-07-30 23:55:38,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/409ccc1344e052df313f0ebb0776f6ff/4=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/5=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/6=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/7=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/8=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/9=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/10=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/11=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/12=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/13=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/14=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/15=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/16=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/17=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/18=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/19=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/20=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/21=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/22=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/23=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/24=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/25=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/26=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/27=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/28=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/29=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/30=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/31=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/32=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/33=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/34=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/35=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/36=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/37=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/38=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/39=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/40=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/41=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/42=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/43=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/44=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/45=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/46=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/47=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/48=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/49=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/50=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/51=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/52=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/53=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/54=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/55=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/56=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/57=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/58=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817, lastWalPosition=6139, nbRowKeys=55, nbHFiles=0, heapSize=11440, lastSeqIds={}, endOfFile=false,usedBufferSize=11440] 2023-07-30 23:55:38,654 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-30 23:55:38,655 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-30 23:55:38,662 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:38,666 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48282, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-30 23:55:38,672 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35865] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:38,672 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35865] zookeeper.ReadOnlyZKClient(139): Connect 0x7000ad3f to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:38,679 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35865] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3c774ea1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:38,689 DEBUG [hconnection-0x212477f2-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,690 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33502, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,721 DEBUG [hconnection-0x212477f2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,730 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48284, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,731 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:38,741 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:38,745 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1729 2023-07-30 23:55:38,745 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 13 WAL entries eligible for replication 2023-07-30 23:55:38,745 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1729 2023-07-30 23:55:38,746 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/4=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/5=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/6=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/7=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/8=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/9=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/10=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/11=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/12=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/13=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/14=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/15=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/16=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=1729, nbRowKeys=13, nbHFiles=0, heapSize=2704, lastSeqIds={}, endOfFile=false,usedBufferSize=2704] 2023-07-30 23:55:38,752 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:38,754 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58108, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-30 23:55:38,755 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:38,756 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] zookeeper.ReadOnlyZKClient(139): Connect 0x44cf25ff to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:38,766 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:38,766 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 45 WAL entries eligible for replication 2023-07-30 23:55:38,766 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:38,775 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@61d94ee4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:38,775 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35865] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:38,777 DEBUG [hconnection-0x75c1c114-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,778 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33518, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,783 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/409ccc1344e052df313f0ebb0776f6ff/59=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/60=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/61=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/62=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/63=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/64=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/65=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/66=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/67=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/68=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/69=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/70=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/71=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/72=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/73=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/74=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/75=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/76=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/77=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/78=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/79=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/80=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/81=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/82=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/83=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/84=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/85=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/86=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/87=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/88=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/89=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/90=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/91=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/92=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/93=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/94=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/95=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/96=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/97=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/98=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/99=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/100=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/101=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/102=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817, lastWalPosition=10975, nbRowKeys=45, nbHFiles=0, heapSize=9360, lastSeqIds={}, endOfFile=false,usedBufferSize=9360] 2023-07-30 23:55:38,791 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:38,794 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:38,799 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54118, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-30 23:55:38,801 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37453] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:38,802 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37453] zookeeper.ReadOnlyZKClient(139): Connect 0x3ba95c86 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:38,825 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37453] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@356eb583, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:38,847 DEBUG [hconnection-0x315137f6-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,848 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33520, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,886 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:38,887 DEBUG [hconnection-0x315137f6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:38,888 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:38,889 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48292, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-30 23:55:38,892 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:38,894 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37453] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:38,901 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8029 2023-07-30 23:55:38,901 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 60 WAL entries eligible for replication 2023-07-30 23:55:38,901 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8029 2023-07-30 23:55:38,901 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/17=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/18=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/19=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/20=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/21=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/22=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/23=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/24=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/25=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/26=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/27=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/28=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/29=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/30=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/31=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/32=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/33=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/34=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/35=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/36=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/37=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/38=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/39=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/40=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/41=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/42=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/43=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/44=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/45=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/46=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/47=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/48=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/49=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/50=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/51=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/52=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/53=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/54=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/55=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/56=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/57=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/58=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/59=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/60=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/61=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/62=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/63=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/64=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/65=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/66=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/67=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/68=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/69=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/70=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/71=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/72=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/73=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/74=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/75=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/76=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=8029, nbRowKeys=60, nbHFiles=0, heapSize=12480, lastSeqIds={}, endOfFile=false,usedBufferSize=12480] 2023-07-30 23:55:38,909 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:38,917 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:38,997 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:38,998 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,006 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,026 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15700 2023-07-30 23:55:39,026 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 72 WAL entries eligible for replication 2023-07-30 23:55:39,026 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15700 2023-07-30 23:55:39,027 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/77=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/78=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/79=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/80=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/81=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/82=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/83=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/84=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/85=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/86=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/87=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/88=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/89=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/90=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/91=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/92=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/93=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/94=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/95=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/96=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/97=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/98=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/99=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/100=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/101=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/102=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/103=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/104=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/105=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/106=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/107=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/108=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/109=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/110=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/111=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/112=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/113=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/114=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/115=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/116=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/117=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/118=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/119=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/120=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/121=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/122=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/123=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/124=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/125=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/126=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/127=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/128=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/129=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/130=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/131=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/132=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/133=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/134=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/135=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/136=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/137=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/138=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/139=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/140=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/141=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/142=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/143=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/144=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/145=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/146=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/147=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/148=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=15700, nbRowKeys=72, nbHFiles=0, heapSize=14976, lastSeqIds={}, endOfFile=false,usedBufferSize=14976] 2023-07-30 23:55:39,031 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:39,038 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43645] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:39,109 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,109 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 55 WAL entries eligible for replication 2023-07-30 23:55:39,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/149=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/150=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/151=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/152=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/153=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/154=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/155=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/156=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/157=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/158=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/159=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/160=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/161=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/162=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/163=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/164=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/165=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/166=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/167=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/168=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/169=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/170=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/171=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/172=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/173=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/174=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/175=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/176=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/177=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/178=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/179=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/180=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/181=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/182=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/183=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/184=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/185=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/186=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/187=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/188=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/189=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/190=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/191=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/192=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/193=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/194=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/195=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/196=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/197=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/198=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/199=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/200=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/201=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/202=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=21752, nbRowKeys=55, nbHFiles=0, heapSize=11440, lastSeqIds={}, endOfFile=false,usedBufferSize=11440] 2023-07-30 23:55:39,144 DEBUG [SinkThread-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:39,146 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48296, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-30 23:55:39,147 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35865] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:39,157 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35865] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:39,219 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,220 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,224 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,250 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,302 INFO [jenkins-hbase4:43687Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:39,308 INFO [jenkins-hbase4:35865Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:39,308 INFO [jenkins-hbase4:35865Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 110 2023-07-30 23:55:39,308 INFO [jenkins-hbase4:43645Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:39,308 INFO [jenkins-hbase4:43645Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 145 2023-07-30 23:55:39,328 INFO [jenkins-hbase4:37453Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:39,329 INFO [jenkins-hbase4:37453Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 45 2023-07-30 23:55:39,331 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,331 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,338 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,355 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,439 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,439 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,444 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,462 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,547 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,547 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,551 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,573 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,592 DEBUG [hconnection-0x7fe37b6d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:39,594 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33522, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:39,599 DEBUG [hconnection-0x7fe37b6d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:39,601 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48304, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:39,624 DEBUG [Listener at localhost/42375] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-30 23:55:39,624 INFO [Listener at localhost/42375] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-30 23:55:39,625 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d6e064e to 127.0.0.1:52585 2023-07-30 23:55:39,625 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,626 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-30 23:55:39,626 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(257): Found active master hash=635847763, stopped=false 2023-07-30 23:55:39,626 INFO [Listener at localhost/42375] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:39,628 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:55:39,628 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:55:39,629 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:55:39,629 INFO [Listener at localhost/42375] procedure2.ProcedureExecutor(629): Stopping 2023-07-30 23:55:39,629 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:39,629 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:55:39,629 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:55:39,629 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5195ded7 to 127.0.0.1:52585 2023-07-30 23:55:39,631 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,631 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35865,1690761333726' ***** 2023-07-30 23:55:39,631 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:55:39,631 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37453,1690761333752' ***** 2023-07-30 23:55:39,631 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:55:39,632 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43687,1690761333781' ***** 2023-07-30 23:55:39,632 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:55:39,632 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43645,1690761333818' ***** 2023-07-30 23:55:39,632 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:55:39,632 INFO [RS:1;jenkins-hbase4:37453] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:55:39,632 INFO [RS:3;jenkins-hbase4:43645] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:55:39,632 INFO [RS:1;jenkins-hbase4:37453] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:55:39,632 INFO [RS:0;jenkins-hbase4:35865] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:55:39,632 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:55:39,632 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:55:39,632 INFO [RS:2;jenkins-hbase4:43687] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:55:39,632 INFO [RS:3;jenkins-hbase4:43645] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:55:39,633 INFO [RS:2;jenkins-hbase4:43687] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:55:39,633 INFO [RS:2;jenkins-hbase4:43687] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:55:39,633 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:55:39,632 INFO [RS:0;jenkins-hbase4:35865] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:55:39,635 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(3305): Received CLOSE for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:39,632 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:55:39,632 INFO [RS:1;jenkins-hbase4:37453] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:55:39,635 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:39,635 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x50e1abb5 to 127.0.0.1:52585 2023-07-30 23:55:39,635 DEBUG [RS:1;jenkins-hbase4:37453] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,635 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37453,1690761333752; all regions closed. 2023-07-30 23:55:39,635 INFO [RS:0;jenkins-hbase4:35865] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:55:39,633 INFO [RS:3;jenkins-hbase4:43645] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:55:39,636 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(3305): Received CLOSE for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:39,636 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:39,636 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c78459e to 127.0.0.1:52585 2023-07-30 23:55:39,636 DEBUG [RS:3;jenkins-hbase4:43645] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,636 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43645,1690761333818; all regions closed. 2023-07-30 23:55:39,642 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:39,642 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55cb9924 to 127.0.0.1:52585 2023-07-30 23:55:39,642 DEBUG [RS:0;jenkins-hbase4:35865] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,642 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-30 23:55:39,642 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1478): Online Regions={6494f975164328a819a48f1beabe91f9=t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.} 2023-07-30 23:55:39,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6494f975164328a819a48f1beabe91f9, disabling compactions & flushes 2023-07-30 23:55:39,643 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(3305): Received CLOSE for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:39,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:39,644 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:39,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:39,644 DEBUG [RS:2;jenkins-hbase4:43687] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x72ad48db to 127.0.0.1:52585 2023-07-30 23:55:39,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. after waiting 0 ms 2023-07-30 23:55:39,644 DEBUG [RS:2;jenkins-hbase4:43687] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:39,645 INFO [RS:2;jenkins-hbase4:43687] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:55:39,645 INFO [RS:2;jenkins-hbase4:43687] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:55:39,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 51a075540234ea8884d31403732b8710, disabling compactions & flushes 2023-07-30 23:55:39,645 INFO [RS:2;jenkins-hbase4:43687] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:55:39,645 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-30 23:55:39,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:39,646 DEBUG [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1504): Waiting on 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:39,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:39,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. after waiting 0 ms 2023-07-30 23:55:39,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:39,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 51a075540234ea8884d31403732b8710 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-30 23:55:39,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6494f975164328a819a48f1beabe91f9 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-30 23:55:39,646 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-30 23:55:39,646 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1478): Online Regions={51a075540234ea8884d31403732b8710=hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., ee738a1d08d169b10fce15cc4dc0d043=t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043., 1588230740=hbase:meta,,1.1588230740} 2023-07-30 23:55:39,646 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1504): Waiting on 1588230740, 51a075540234ea8884d31403732b8710, ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:39,648 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:55:39,648 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:55:39,648 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:55:39,648 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:55:39,649 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:55:39,651 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-30 23:55:39,660 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,660 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,660 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,663 DEBUG [RS:1;jenkins-hbase4:37453] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:39,663 INFO [RS:1;jenkins-hbase4:37453] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37453%2C1690761333752.jenkins-hbase4.apache.org%2C37453%2C1690761333752.regiongroup-0:(num 1690761334447) 2023-07-30 23:55:39,663 DEBUG [RS:1;jenkins-hbase4:37453] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,664 INFO [RS:1;jenkins-hbase4:37453] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,664 DEBUG [RS:3;jenkins-hbase4:43645] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:39,664 INFO [RS:3;jenkins-hbase4:43645] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43645%2C1690761333818.jenkins-hbase4.apache.org%2C43645%2C1690761333818.regiongroup-0:(num 1690761334435) 2023-07-30 23:55:39,664 DEBUG [RS:3;jenkins-hbase4:43645] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,664 INFO [RS:3;jenkins-hbase4:43645] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,664 INFO [RS:1;jenkins-hbase4:37453] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-30 23:55:39,664 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:39,664 INFO [RS:3;jenkins-hbase4:43645] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-30 23:55:39,664 INFO [RS:1;jenkins-hbase4:37453] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:55:39,665 INFO [RS:1;jenkins-hbase4:37453] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:55:39,665 INFO [RS:1;jenkins-hbase4:37453] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:55:39,665 DEBUG [RS:1;jenkins-hbase4:37453] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ba95c86 to 127.0.0.1:52585 2023-07-30 23:55:39,665 DEBUG [RS:1;jenkins-hbase4:37453] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,667 INFO [RS:3;jenkins-hbase4:43645] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:55:39,667 INFO [RS:1;jenkins-hbase4:37453] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37453 2023-07-30 23:55:39,667 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:39,668 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,667 INFO [RS:3;jenkins-hbase4:43645] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:55:39,668 INFO [RS:3;jenkins-hbase4:43645] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:55:39,668 DEBUG [RS:3;jenkins-hbase4:43645] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x44cf25ff to 127.0.0.1:52585 2023-07-30 23:55:39,669 DEBUG [RS:3;jenkins-hbase4:43645] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,670 INFO [RS:3;jenkins-hbase4:43645] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43645 2023-07-30 23:55:39,684 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,684 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,686 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,688 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,734 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:39,734 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,734 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:39,734 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,734 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,735 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:39,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:39,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:39,735 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:39,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:39,739 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:39,739 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:39,739 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90018, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:39,739 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,37453,1690761333752 2023-07-30 23:55:39,739 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:39,739 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90016, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,739 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:39,739 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43645,1690761333818 2023-07-30 23:55:39,741 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:55:39,742 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:55:39,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/.tmp/cf1/00173fc536fb4d1b8a36c88d95b68e51 2023-07-30 23:55:39,752 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/info/11fef2fd43a54b52a3a56fd6edead3d8 2023-07-30 23:55:39,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/.tmp/info/fba57fb05b8f46a0a780d0a7c0f6febf 2023-07-30 23:55:39,766 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,766 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,768 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,796 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/.tmp/cf1/00173fc536fb4d1b8a36c88d95b68e51 as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/00173fc536fb4d1b8a36c88d95b68e51 2023-07-30 23:55:39,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/.tmp/info/fba57fb05b8f46a0a780d0a7c0f6febf as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info/fba57fb05b8f46a0a780d0a7c0f6febf 2023-07-30 23:55:39,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info/fba57fb05b8f46a0a780d0a7c0f6febf, entries=2, sequenceid=6, filesize=4.8 K 2023-07-30 23:55:39,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 51a075540234ea8884d31403732b8710 in 174ms, sequenceid=6, compaction requested=false 2023-07-30 23:55:39,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/00173fc536fb4d1b8a36c88d95b68e51, entries=100, sequenceid=6, filesize=8.3 K 2023-07-30 23:55:39,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 6494f975164328a819a48f1beabe91f9 in 178ms, sequenceid=6, compaction requested=false 2023-07-30 23:55:39,838 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/table/1f119aa9b99642ff9ae26cb85132b1b4 2023-07-30 23:55:39,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-30 23:55:39,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-30 23:55:39,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:39,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:55:39,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:39,842 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:39,842 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:55:39,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:39,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee738a1d08d169b10fce15cc4dc0d043, disabling compactions & flushes 2023-07-30 23:55:39,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:39,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:39,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. after waiting 0 ms 2023-07-30 23:55:39,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:39,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ee738a1d08d169b10fce15cc4dc0d043 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-30 23:55:39,846 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35865,1690761333726; all regions closed. 2023-07-30 23:55:39,846 DEBUG [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1504): Waiting on 1588230740, ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:39,854 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/info/11fef2fd43a54b52a3a56fd6edead3d8 as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/11fef2fd43a54b52a3a56fd6edead3d8 2023-07-30 23:55:39,856 DEBUG [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:39,856 INFO [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35865%2C1690761333726.jenkins-hbase4.apache.org%2C35865%2C1690761333726.regiongroup-1:(num 1690761337040) 2023-07-30 23:55:39,864 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/11fef2fd43a54b52a3a56fd6edead3d8, entries=30, sequenceid=19, filesize=8.1 K 2023-07-30 23:55:39,864 DEBUG [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:39,864 INFO [RS:0;jenkins-hbase4:35865] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35865%2C1690761333726.jenkins-hbase4.apache.org%2C35865%2C1690761333726.regiongroup-0:(num 1690761334433) 2023-07-30 23:55:39,864 DEBUG [RS:0;jenkins-hbase4:35865] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,864 INFO [RS:0;jenkins-hbase4:35865] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:39,864 INFO [RS:0;jenkins-hbase4:35865] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-30 23:55:39,865 INFO [RS:0;jenkins-hbase4:35865] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:55:39,865 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:39,865 INFO [RS:0;jenkins-hbase4:35865] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:55:39,865 INFO [RS:0;jenkins-hbase4:35865] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:55:39,865 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/table/1f119aa9b99642ff9ae26cb85132b1b4 as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table/1f119aa9b99642ff9ae26cb85132b1b4 2023-07-30 23:55:39,865 DEBUG [RS:0;jenkins-hbase4:35865] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7000ad3f to 127.0.0.1:52585 2023-07-30 23:55:39,865 DEBUG [RS:0;jenkins-hbase4:35865] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:39,867 INFO [RS:0;jenkins-hbase4:35865] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35865 2023-07-30 23:55:39,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/.tmp/cf1/39f56f6009014b15a197bd88c2d5fadc 2023-07-30 23:55:39,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,879 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table/1f119aa9b99642ff9ae26cb85132b1b4, entries=6, sequenceid=19, filesize=4.8 K 2023-07-30 23:55:39,880 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:39,880 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/.tmp/cf1/39f56f6009014b15a197bd88c2d5fadc as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/39f56f6009014b15a197bd88c2d5fadc 2023-07-30 23:55:39,884 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 233ms, sequenceid=19, compaction requested=false 2023-07-30 23:55:39,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-30 23:55:39,894 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-30 23:55:39,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:55:39,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:55:39,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-30 23:55:39,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/39f56f6009014b15a197bd88c2d5fadc, entries=200, sequenceid=8, filesize=12.2 K 2023-07-30 23:55:39,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for ee738a1d08d169b10fce15cc4dc0d043 in 53ms, sequenceid=8, compaction requested=false 2023-07-30 23:55:39,903 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:39,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-07-30 23:55:39,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:39,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:55:39,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:39,982 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:39,985 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:39,985 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,008 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,038 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43645,1690761333818] 2023-07-30 23:55:40,038 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43645,1690761333818; numProcessing=1 2023-07-30 23:55:40,038 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90018, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:40,038 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:40,038 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:55:40,038 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90016, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:40,038 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,35865,1690761333726 2023-07-30 23:55:40,038 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:40,038 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:55:40,039 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43645,1690761333818 already deleted, retry=false 2023-07-30 23:55:40,039 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43645,1690761333818 expired; onlineServers=3 2023-07-30 23:55:40,039 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37453,1690761333752] 2023-07-30 23:55:40,040 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37453,1690761333752; numProcessing=2 2023-07-30 23:55:40,042 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,37453,1690761333752 already deleted, retry=false 2023-07-30 23:55:40,042 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37453,1690761333752 expired; onlineServers=2 2023-07-30 23:55:40,042 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35865,1690761333726] 2023-07-30 23:55:40,042 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35865,1690761333726; numProcessing=3 2023-07-30 23:55:40,043 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,35865,1690761333726 already deleted, retry=false 2023-07-30 23:55:40,043 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35865,1690761333726 expired; onlineServers=1 2023-07-30 23:55:40,047 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43687,1690761333781; all regions closed. 2023-07-30 23:55:40,054 DEBUG [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:40,054 INFO [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43687%2C1690761333781.meta:.meta(num 1690761334515) 2023-07-30 23:55:40,063 DEBUG [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:40,063 INFO [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43687%2C1690761333781.jenkins-hbase4.apache.org%2C43687%2C1690761333781.regiongroup-0:(num 1690761334415) 2023-07-30 23:55:40,070 DEBUG [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:55:40,070 INFO [RS:2;jenkins-hbase4:43687] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43687%2C1690761333781.jenkins-hbase4.apache.org%2C43687%2C1690761333781.regiongroup-1:(num 1690761335076) 2023-07-30 23:55:40,070 DEBUG [RS:2;jenkins-hbase4:43687] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:40,070 INFO [RS:2;jenkins-hbase4:43687] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:40,070 INFO [RS:2;jenkins-hbase4:43687] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-30 23:55:40,070 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:40,071 INFO [RS:2;jenkins-hbase4:43687] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43687 2023-07-30 23:55:40,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90016, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:40,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90018, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:40,073 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:55:40,073 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:40,073 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:55:40,073 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43687,1690761333781 2023-07-30 23:55:40,074 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43687,1690761333781] 2023-07-30 23:55:40,074 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43687,1690761333781; numProcessing=4 2023-07-30 23:55:40,076 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43687,1690761333781 already deleted, retry=false 2023-07-30 23:55:40,076 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43687,1690761333781 expired; onlineServers=0 2023-07-30 23:55:40,076 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40341,1690761333687' ***** 2023-07-30 23:55:40,076 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-30 23:55:40,077 DEBUG [M:0;jenkins-hbase4:40341] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21d9c437, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:40,077 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40341,1690761333687 2023-07-30 23:55:40,077 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40341,1690761333687; all regions closed. 2023-07-30 23:55:40,077 DEBUG [M:0;jenkins-hbase4:40341] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:40,077 DEBUG [M:0;jenkins-hbase4:40341] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-30 23:55:40,078 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-30 23:55:40,078 DEBUG [M:0;jenkins-hbase4:40341] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-30 23:55:40,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761334066] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761334066,5,FailOnTimeoutGroup] 2023-07-30 23:55:40,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761334066] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761334066,5,FailOnTimeoutGroup] 2023-07-30 23:55:40,078 INFO [M:0;jenkins-hbase4:40341] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-30 23:55:40,080 INFO [M:0;jenkins-hbase4:40341] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-30 23:55:40,080 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-30 23:55:40,080 INFO [M:0;jenkins-hbase4:40341] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-30 23:55:40,080 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:40,080 DEBUG [M:0;jenkins-hbase4:40341] master.HMaster(1512): Stopping service threads 2023-07-30 23:55:40,080 INFO [M:0;jenkins-hbase4:40341] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-30 23:55:40,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:40,081 ERROR [M:0;jenkins-hbase4:40341] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-30 23:55:40,081 INFO [M:0;jenkins-hbase4:40341] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-30 23:55:40,081 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-30 23:55:40,082 DEBUG [M:0;jenkins-hbase4:40341] zookeeper.ZKUtil(398): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-30 23:55:40,082 WARN [M:0;jenkins-hbase4:40341] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-30 23:55:40,082 INFO [M:0;jenkins-hbase4:40341] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-30 23:55:40,082 INFO [M:0;jenkins-hbase4:40341] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-30 23:55:40,083 DEBUG [M:0;jenkins-hbase4:40341] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:55:40,083 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:40,083 DEBUG [M:0;jenkins-hbase4:40341] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:40,083 DEBUG [M:0;jenkins-hbase4:40341] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:55:40,083 DEBUG [M:0;jenkins-hbase4:40341] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:40,083 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.39 KB heapSize=65.18 KB 2023-07-30 23:55:40,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,090 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,090 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,104 INFO [M:0;jenkins-hbase4:40341] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.39 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/906dafa714534cac8f768868161da0a3 2023-07-30 23:55:40,111 DEBUG [M:0;jenkins-hbase4:40341] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/906dafa714534cac8f768868161da0a3 as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/906dafa714534cac8f768868161da0a3 2023-07-30 23:55:40,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,117 INFO [M:0;jenkins-hbase4:40341] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/906dafa714534cac8f768868161da0a3, entries=14, sequenceid=116, filesize=8.1 K 2023-07-30 23:55:40,118 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegion(2948): Finished flush of dataSize ~55.39 KB/56720, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 35ms, sequenceid=116, compaction requested=false 2023-07-30 23:55:40,120 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:40,120 DEBUG [M:0;jenkins-hbase4:40341] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:40,126 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:40,126 INFO [M:0;jenkins-hbase4:40341] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-30 23:55:40,127 INFO [M:0;jenkins-hbase4:40341] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40341 2023-07-30 23:55:40,129 DEBUG [M:0;jenkins-hbase4:40341] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,40341,1690761333687 already deleted, retry=false 2023-07-30 23:55:40,195 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,195 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,195 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,217 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,245 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,245 INFO [M:0;jenkins-hbase4:40341] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40341,1690761333687; zookeeper connection closed. 2023-07-30 23:55:40,245 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:40341-0x101b8a8e9d90007, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,300 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,301 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,301 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,322 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,345 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,345 INFO [RS:2;jenkins-hbase4:43687] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43687,1690761333781; zookeeper connection closed. 2023-07-30 23:55:40,345 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43687-0x101b8a8e9d9000a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,346 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2755cac5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2755cac5 2023-07-30 23:55:40,406 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,406 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,406 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,427 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,445 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,445 INFO [RS:0;jenkins-hbase4:35865] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35865,1690761333726; zookeeper connection closed. 2023-07-30 23:55:40,445 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:35865-0x101b8a8e9d90008, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,446 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@715cf8b4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@715cf8b4 2023-07-30 23:55:40,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,531 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,617 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,617 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,617 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,636 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,645 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,645 INFO [RS:1;jenkins-hbase4:37453] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37453,1690761333752; zookeeper connection closed. 2023-07-30 23:55:40,645 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:37453-0x101b8a8e9d90009, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,647 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@61b85b45] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@61b85b45 2023-07-30 23:55:40,722 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-30 23:55:40,722 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,722 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-30 23:55:40,741 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-30 23:55:40,745 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,745 INFO [RS:3;jenkins-hbase4:43645] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43645,1690761333818; zookeeper connection closed. 2023-07-30 23:55:40,746 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43645-0x101b8a8e9d9000b, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:40,747 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1d6a99d4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1d6a99d4 2023-07-30 23:55:40,748 INFO [Listener at localhost/42375] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-30 23:55:40,775 INFO [Listener at localhost/42375] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-30 23:55:40,776 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26a35432 to 127.0.0.1:52585 2023-07-30 23:55:40,776 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:40,776 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-30 23:55:40,776 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(257): Found active master hash=346379769, stopped=false 2023-07-30 23:55:40,776 INFO [Listener at localhost/42375] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:40,779 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-30 23:55:40,779 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-30 23:55:40,779 INFO [Listener at localhost/42375] procedure2.ProcedureExecutor(629): Stopping 2023-07-30 23:55:40,779 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-30 23:55:40,779 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e15bf89 to 127.0.0.1:52585 2023-07-30 23:55:40,779 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:40,782 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:40,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:55:40,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:55:40,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:55:40,782 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42313,1690761329724' ***** 2023-07-30 23:55:40,782 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:55:40,782 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36237,1690761329770' ***** 2023-07-30 23:55:40,782 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:55:40,782 INFO [RS:0;jenkins-hbase4:42313] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:55:40,783 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:55:40,783 INFO [RS:1;jenkins-hbase4:36237] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:55:40,783 INFO [RS:0;jenkins-hbase4:42313] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:55:40,784 INFO [RS:0;jenkins-hbase4:42313] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:55:40,784 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(3305): Received CLOSE for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:40,783 INFO [RS:1;jenkins-hbase4:36237] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:55:40,784 INFO [RS:1;jenkins-hbase4:36237] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:55:40,784 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(3305): Received CLOSE for bffda5817d519b05008d136099bf920f 2023-07-30 23:55:40,784 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(3305): Received CLOSE for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:40,783 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:55:40,784 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:40,785 DEBUG [RS:0;jenkins-hbase4:42313] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e7a8c53 to 127.0.0.1:52585 2023-07-30 23:55:40,785 DEBUG [RS:0;jenkins-hbase4:42313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:40,785 INFO [RS:0;jenkins-hbase4:42313] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:55:40,785 INFO [RS:0;jenkins-hbase4:42313] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:55:40,785 INFO [RS:0;jenkins-hbase4:42313] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:55:40,785 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-30 23:55:40,785 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:40,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 409ccc1344e052df313f0ebb0776f6ff, disabling compactions & flushes 2023-07-30 23:55:40,786 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:40,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ef4a62973dc7531db8626f8122ae55c2, disabling compactions & flushes 2023-07-30 23:55:40,785 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-30 23:55:40,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:40,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:40,787 DEBUG [RS:1;jenkins-hbase4:36237] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x443a81ac to 127.0.0.1:52585 2023-07-30 23:55:40,786 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:55:40,788 DEBUG [RS:1;jenkins-hbase4:36237] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:40,788 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:55:40,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:40,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:40,787 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1478): Online Regions={ef4a62973dc7531db8626f8122ae55c2=hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2., 1588230740=hbase:meta,,1.1588230740, bffda5817d519b05008d136099bf920f=t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.} 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. after waiting 0 ms 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. after waiting 0 ms 2023-07-30 23:55:40,788 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1504): Waiting on 1588230740, bffda5817d519b05008d136099bf920f, ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:55:40,788 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:55:40,788 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1478): Online Regions={409ccc1344e052df313f0ebb0776f6ff=t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.} 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:40,788 DEBUG [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1504): Waiting on 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:55:40,788 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-30 23:55:40,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 409ccc1344e052df313f0ebb0776f6ff 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-30 23:55:40,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:40,788 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:40,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ef4a62973dc7531db8626f8122ae55c2 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-30 23:55:40,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/cf1/a3a52480850d45169a167bf2c35045a6 2023-07-30 23:55:40,830 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-07-30 23:55:40,830 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:40,830 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141, lastWalPosition=845, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:40,830 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-07-30 23:55:40,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/.tmp/info/d7b0678a304d4276ba635aae66502e80 2023-07-30 23:55:40,831 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a3a52480850d45169a167bf2c35045a6 2023-07-30 23:55:40,835 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-30 23:55:40,835 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-30 23:55:40,835 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-30 23:55:40,835 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/409ccc1344e052df313f0ebb0776f6ff/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817, lastWalPosition=14562, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-30 23:55:40,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/.tmp/info/d7b0678a304d4276ba635aae66502e80 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info/d7b0678a304d4276ba635aae66502e80 2023-07-30 23:55:40,849 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-07-30 23:55:40,850 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-30 23:55:40,850 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-07-30 23:55:40,850 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=28400, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-30 23:55:40,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info/d7b0678a304d4276ba635aae66502e80, entries=2, sequenceid=6, filesize=4.8 K 2023-07-30 23:55:40,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for ef4a62973dc7531db8626f8122ae55c2 in 69ms, sequenceid=6, compaction requested=false 2023-07-30 23:55:40,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/norep/d50180c3272d4b76b6cde45b0d81dbbc 2023-07-30 23:55:40,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-30 23:55:40,865 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:40,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ef4a62973dc7531db8626f8122ae55c2: 2023-07-30 23:55:40,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:55:40,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bffda5817d519b05008d136099bf920f, disabling compactions & flushes 2023-07-30 23:55:40,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:40,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:40,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. after waiting 0 ms 2023-07-30 23:55:40,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:40,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing bffda5817d519b05008d136099bf920f 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-30 23:55:40,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d50180c3272d4b76b6cde45b0d81dbbc 2023-07-30 23:55:40,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/cf1/a3a52480850d45169a167bf2c35045a6 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1/a3a52480850d45169a167bf2c35045a6 2023-07-30 23:55:40,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a3a52480850d45169a167bf2c35045a6 2023-07-30 23:55:40,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1/a3a52480850d45169a167bf2c35045a6, entries=100, sequenceid=106, filesize=8.2 K 2023-07-30 23:55:40,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/norep/d50180c3272d4b76b6cde45b0d81dbbc as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep/d50180c3272d4b76b6cde45b0d81dbbc 2023-07-30 23:55:40,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d50180c3272d4b76b6cde45b0d81dbbc 2023-07-30 23:55:40,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep/d50180c3272d4b76b6cde45b0d81dbbc, entries=51, sequenceid=106, filesize=6.5 K 2023-07-30 23:55:40,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for 409ccc1344e052df313f0ebb0776f6ff in 97ms, sequenceid=106, compaction requested=false 2023-07-30 23:55:40,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/cf1/f91a173b57f14bb9b2758966745afc1d 2023-07-30 23:55:40,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-30 23:55:40,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:40,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 409ccc1344e052df313f0ebb0776f6ff: 2023-07-30 23:55:40,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:55:40,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f91a173b57f14bb9b2758966745afc1d 2023-07-30 23:55:40,919 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/norep/867abfcf8f67400aba2b3ee4fd343393 2023-07-30 23:55:40,925 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 867abfcf8f67400aba2b3ee4fd343393 2023-07-30 23:55:40,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/cf1/f91a173b57f14bb9b2758966745afc1d as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1/f91a173b57f14bb9b2758966745afc1d 2023-07-30 23:55:40,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f91a173b57f14bb9b2758966745afc1d 2023-07-30 23:55:40,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1/f91a173b57f14bb9b2758966745afc1d, entries=200, sequenceid=206, filesize=11.9 K 2023-07-30 23:55:40,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/norep/867abfcf8f67400aba2b3ee4fd343393 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep/867abfcf8f67400aba2b3ee4fd343393 2023-07-30 23:55:40,941 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:40,941 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:40,941 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:40,941 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:55:40,942 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-30 23:55:40,942 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:40,942 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-30 23:55:40,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 867abfcf8f67400aba2b3ee4fd343393 2023-07-30 23:55:40,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep/867abfcf8f67400aba2b3ee4fd343393, entries=101, sequenceid=206, filesize=8.3 K 2023-07-30 23:55:40,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for bffda5817d519b05008d136099bf920f in 81ms, sequenceid=206, compaction requested=false 2023-07-30 23:55:40,957 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:40,957 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:40,957 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:40,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-30 23:55:40,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:40,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bffda5817d519b05008d136099bf920f: 2023-07-30 23:55:40,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:55:40,971 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-30 23:55:40,971 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-30 23:55:40,971 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-30 23:55:40,971 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-30 23:55:40,988 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-30 23:55:40,988 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36237,1690761329770; all regions closed. 2023-07-30 23:55:40,995 DEBUG [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:55:40,996 INFO [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0:(num 1690761331137) 2023-07-30 23:55:41,001 DEBUG [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:55:41,001 INFO [RS:1;jenkins-hbase4:36237] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1:(num 1690761335817) 2023-07-30 23:55:41,001 DEBUG [RS:1;jenkins-hbase4:36237] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:41,002 INFO [RS:1;jenkins-hbase4:36237] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:41,002 INFO [RS:1;jenkins-hbase4:36237] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-30 23:55:41,002 INFO [RS:1;jenkins-hbase4:36237] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:55:41,002 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:41,002 INFO [RS:1;jenkins-hbase4:36237] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:55:41,002 INFO [RS:1;jenkins-hbase4:36237] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:55:41,002 INFO [RS:1;jenkins-hbase4:36237] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-30 23:55:41,013 INFO [jenkins-hbase4:42313Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=8120B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 200, current progress: walGroup [jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0]: currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 at position: 21752 walGroup [jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1]: currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141 at position: 845 2023-07-30 23:55:41,014 INFO [jenkins-hbase4:36237Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=4120B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 100, current progress: walGroup [jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1]: currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817 at position: 10975 walGroup [jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0]: currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 at position: -1 2023-07-30 23:55:41,043 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 was moved to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 2023-07-30 23:55:41,044 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817 was moved to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817 2023-07-30 23:55:41,046 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:41,061 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:41,153 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:41,167 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:41,188 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-30 23:55:41,216 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/info/4b3131ff2cbb4bf289d46c3f198185af 2023-07-30 23:55:41,234 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/rep_barrier/6e9a76720bdb4beb853377b05d4385f2 2023-07-30 23:55:41,248 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/table/d18717fd499f4adbb84d85844f86d4bc 2023-07-30 23:55:41,254 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/info/4b3131ff2cbb4bf289d46c3f198185af as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info/4b3131ff2cbb4bf289d46c3f198185af 2023-07-30 23:55:41,258 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:41,261 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info/4b3131ff2cbb4bf289d46c3f198185af, entries=30, sequenceid=19, filesize=8.1 K 2023-07-30 23:55:41,262 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/rep_barrier/6e9a76720bdb4beb853377b05d4385f2 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier/6e9a76720bdb4beb853377b05d4385f2 2023-07-30 23:55:41,270 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier/6e9a76720bdb4beb853377b05d4385f2, entries=2, sequenceid=19, filesize=4.9 K 2023-07-30 23:55:41,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/table/d18717fd499f4adbb84d85844f86d4bc as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table/d18717fd499f4adbb84d85844f86d4bc 2023-07-30 23:55:41,271 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:41,279 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90018, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:41,279 DEBUG [RS:1;jenkins-hbase4:36237] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d00cd60 to 127.0.0.1:52585 2023-07-30 23:55:41,279 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90018, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:41,279 DEBUG [RS:1;jenkins-hbase4:36237] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:41,284 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table/d18717fd499f4adbb84d85844f86d4bc, entries=6, sequenceid=19, filesize=4.8 K 2023-07-30 23:55:41,285 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 497ms, sequenceid=19, compaction requested=false 2023-07-30 23:55:41,383 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:381) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) 2023-07-30 23:55:41,383 INFO [RS:1;jenkins-hbase4:36237] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1 terminated 2023-07-30 23:55:41,385 INFO [RS:1;jenkins-hbase4:36237] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1 terminated 2023-07-30 23:55:41,386 INFO [RS:1;jenkins-hbase4:36237] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36237 2023-07-30 23:55:41,389 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-30 23:55:41,389 DEBUG [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-30 23:55:41,390 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:41,390 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-30 23:55:41,391 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:55:41,391 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:55:41,391 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-30 23:55:41,391 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:41,391 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:41,391 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:55:41,391 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:55:41,391 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:55:41,392 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:41,393 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36237,1690761329770] 2023-07-30 23:55:41,393 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36237,1690761329770; numProcessing=1 2023-07-30 23:55:41,394 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,36237,1690761329770 already deleted, retry=false 2023-07-30 23:55:41,394 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36237,1690761329770 expired; onlineServers=1 2023-07-30 23:55:41,493 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:41,493 INFO [RS:1;jenkins-hbase4:36237] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36237,1690761329770; zookeeper connection closed. 2023-07-30 23:55:41,493 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:36237-0x101b8a8e9d90002, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:41,493 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7dc0103b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7dc0103b 2023-07-30 23:55:41,497 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:41,497 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-30 23:55:41,590 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42313,1690761329724; all regions closed. 2023-07-30 23:55:41,596 DEBUG [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:55:41,596 INFO [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42313%2C1690761329724.meta:.meta(num 1690761331371) 2023-07-30 23:55:41,602 DEBUG [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:55:41,602 INFO [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0:(num 1690761331137) 2023-07-30 23:55:41,604 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-30 23:55:41,609 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.ProtobufLogReader(425): Encountered a malformed edit, seeking back to last good position in file, from 29266 to 29258 java.io.EOFException: Invalid PB, EOF? Ignoring; originalPosition=29258, currentPosition=29266 at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:354) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:95) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:83) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.readNextEntryAndRecordReaderPosition(WALEntryStream.java:259) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:173) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:102) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.tryAdvanceStreamAndCreateWALBatch(ReplicationSourceWALReader.java:259) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:145) Caused by: org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: encoded_region_name, table_name, log_sequence_number, write_time at org.apache.hbase.thirdparty.com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:79) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.checkMessageInitialized(AbstractParser.java:68) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:232) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:237) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.parseDelimitedFrom(ProtobufUtil.java:3578) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:348) ... 7 more 2023-07-30 23:55:41,610 DEBUG [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:55:41,610 INFO [RS:0;jenkins-hbase4:42313] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1:(num 1690761332141) 2023-07-30 23:55:41,610 DEBUG [RS:0;jenkins-hbase4:42313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:41,610 INFO [RS:0;jenkins-hbase4:42313] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:55:41,610 INFO [RS:0;jenkins-hbase4:42313] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-30 23:55:41,610 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:41,610 INFO [RS:0;jenkins-hbase4:42313] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-30 23:55:41,610 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 was moved to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 2023-07-30 23:55:41,612 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141, it was closed cleanly currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141 at position: 1430 2023-07-30 23:55:41,612 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141 2023-07-30 23:55:41,612 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:41,614 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 2023-07-30 23:55:41,614 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:41,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90016, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:41,714 DEBUG [RS:0;jenkins-hbase4:42313] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2703aa98 to 127.0.0.1:52585 2023-07-30 23:55:41,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90016, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:41,714 DEBUG [RS:0;jenkins-hbase4:42313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:41,914 INFO [RS:0;jenkins-hbase4:42313] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1 terminated 2023-07-30 23:55:41,915 INFO [RS:0;jenkins-hbase4:42313] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1 terminated 2023-07-30 23:55:41,916 INFO [RS:0;jenkins-hbase4:42313] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42313 2023-07-30 23:55:41,918 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:41,918 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:55:41,919 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:41,920 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42313,1690761329724; numProcessing=2 2023-07-30 23:55:41,922 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42313,1690761329724 already deleted, retry=false 2023-07-30 23:55:41,922 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42313,1690761329724 expired; onlineServers=0 2023-07-30 23:55:41,922 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43527,1690761328479' ***** 2023-07-30 23:55:41,922 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-30 23:55:41,923 DEBUG [M:0;jenkins-hbase4:43527] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55985fc2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:41,923 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43527,1690761328479 2023-07-30 23:55:41,923 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43527,1690761328479; all regions closed. 2023-07-30 23:55:41,923 DEBUG [M:0;jenkins-hbase4:43527] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:41,923 DEBUG [M:0;jenkins-hbase4:43527] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-30 23:55:41,923 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-30 23:55:41,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761330829] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761330829,5,FailOnTimeoutGroup] 2023-07-30 23:55:41,923 DEBUG [M:0;jenkins-hbase4:43527] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-30 23:55:41,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761330825] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761330825,5,FailOnTimeoutGroup] 2023-07-30 23:55:41,925 INFO [M:0;jenkins-hbase4:43527] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-30 23:55:41,925 INFO [M:0;jenkins-hbase4:43527] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-30 23:55:41,925 INFO [M:0;jenkins-hbase4:43527] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-30 23:55:41,925 DEBUG [M:0;jenkins-hbase4:43527] master.HMaster(1512): Stopping service threads 2023-07-30 23:55:41,925 INFO [M:0;jenkins-hbase4:43527] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-30 23:55:41,925 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-30 23:55:41,926 ERROR [M:0;jenkins-hbase4:43527] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-30 23:55:41,926 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:55:41,926 INFO [M:0;jenkins-hbase4:43527] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-30 23:55:41,926 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-30 23:55:41,927 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:55:41,927 DEBUG [M:0;jenkins-hbase4:43527] zookeeper.ZKUtil(398): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-30 23:55:41,927 WARN [M:0;jenkins-hbase4:43527] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-30 23:55:41,927 INFO [M:0;jenkins-hbase4:43527] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-30 23:55:41,928 INFO [M:0;jenkins-hbase4:43527] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-30 23:55:41,928 DEBUG [M:0;jenkins-hbase4:43527] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:55:41,928 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:41,929 DEBUG [M:0;jenkins-hbase4:43527] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:41,929 DEBUG [M:0;jenkins-hbase4:43527] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:55:41,929 DEBUG [M:0;jenkins-hbase4:43527] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:41,929 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.02 KB heapSize=72.60 KB 2023-07-30 23:55:41,941 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:55:41,943 INFO [M:0;jenkins-hbase4:43527] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.02 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c20b0c3094344a888ac1fd92ef271752 2023-07-30 23:55:41,949 DEBUG [M:0;jenkins-hbase4:43527] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c20b0c3094344a888ac1fd92ef271752 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c20b0c3094344a888ac1fd92ef271752 2023-07-30 23:55:41,955 INFO [M:0;jenkins-hbase4:43527] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c20b0c3094344a888ac1fd92ef271752, entries=17, sequenceid=136, filesize=8.5 K 2023-07-30 23:55:41,956 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegion(2948): Finished flush of dataSize ~61.02 KB/62484, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=136, compaction requested=false 2023-07-30 23:55:41,958 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:55:41,958 DEBUG [M:0;jenkins-hbase4:43527] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:41,962 INFO [M:0;jenkins-hbase4:43527] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-30 23:55:41,962 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:55:41,963 INFO [M:0;jenkins-hbase4:43527] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43527 2023-07-30 23:55:41,964 DEBUG [M:0;jenkins-hbase4:43527] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,43527,1690761328479 already deleted, retry=false 2023-07-30 23:55:42,020 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:42,020 INFO [RS:0;jenkins-hbase4:42313] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42313,1690761329724; zookeeper connection closed. 2023-07-30 23:55:42,020 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): regionserver:42313-0x101b8a8e9d90001, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:42,020 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@19dd86aa] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@19dd86aa 2023-07-30 23:55:42,020 INFO [Listener at localhost/42375] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-30 23:55:42,120 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:42,120 INFO [M:0;jenkins-hbase4:43527] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43527,1690761328479; zookeeper connection closed. 2023-07-30 23:55:42,120 DEBUG [Listener at localhost/45953-EventThread] zookeeper.ZKWatcher(600): master:43527-0x101b8a8e9d90000, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:55:42,123 INFO [Listener at localhost/42375] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:42,123 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:42,124 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:42,124 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:42,124 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:42,124 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:42,124 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:42,125 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37557 2023-07-30 23:55:42,125 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:42,126 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:42,127 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37557 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:42,134 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:375570x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:42,135 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37557-0x101b8a8e9d9001c connected 2023-07-30 23:55:42,138 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:42,138 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:42,139 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:42,139 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37557 2023-07-30 23:55:42,139 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37557 2023-07-30 23:55:42,140 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37557 2023-07-30 23:55:42,140 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37557 2023-07-30 23:55:42,140 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37557 2023-07-30 23:55:42,140 INFO [Listener at localhost/42375] master.HMaster(444): hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc, hbase.cluster.distributed=false 2023-07-30 23:55:42,141 DEBUG [pool-456-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:55:42,153 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:55:42,154 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46759 2023-07-30 23:55:42,154 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:55:42,156 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:55:42,156 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:42,157 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:42,158 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46759 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:42,161 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:467590x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:42,162 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:467590x0, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:55:42,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46759-0x101b8a8e9d9001d connected 2023-07-30 23:55:42,163 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:55:42,164 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:55:42,164 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46759 2023-07-30 23:55:42,164 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46759 2023-07-30 23:55:42,164 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46759 2023-07-30 23:55:42,165 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46759 2023-07-30 23:55:42,165 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46759 2023-07-30 23:55:42,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,167 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-30 23:55:42,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,171 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:42,171 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:55:42,171 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:42,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-30 23:55:42,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-30 23:55:42,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,37557,1690761342123 from backup master directory 2023-07-30 23:55:42,174 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,174 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-30 23:55:42,174 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:42,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:55:42,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x68f3d9bf to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:42,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4cb0caf7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:42,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:55:42,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-30 23:55:42,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:42,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:42,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687 to hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687-dead as it is dead 2023-07-30 23:55:42,231 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687-dead/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 2023-07-30 23:55:42,235 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687-dead/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 after 4ms 2023-07-30 23:55:42,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687-dead/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 to hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 2023-07-30 23:55:42,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,40341,1690761333687-dead 2023-07-30 23:55:42,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:42,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/oldWALs, maxLogs=10 2023-07-30 23:55:42,254 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:42,254 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:42,254 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:42,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 2023-07-30 23:55:42,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:42,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:42,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:42,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:42,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:42,259 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:42,260 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-30 23:55:42,260 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-30 23:55:42,267 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/906dafa714534cac8f768868161da0a3 2023-07-30 23:55:42,268 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:42,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-30 23:55:42,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 2023-07-30 23:55:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 2023-07-30 23:55:42,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40341%2C1690761333687.jenkins-hbase4.apache.org%2C40341%2C1690761333687.regiongroup-0.1690761333960 2023-07-30 23:55:42,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:55:42,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-30 23:55:42,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63330862, jitterRate=-0.05629661679267883}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:55:42,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:55:42,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-30 23:55:42,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-30 23:55:42,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-30 23:55:42,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-30 23:55:42,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-30 23:55:42,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-30 23:55:42,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-30 23:55:42,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-30 23:55:42,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-30 23:55:42,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-30 23:55:42,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-30 23:55:42,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-07-30 23:55:42,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-30 23:55:42,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-30 23:55:42,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43687,1690761333781, table=hbase:meta, region=1588230740 2023-07-30 23:55:42,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-30 23:55:42,305 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43645,1690761333818 already deleted, retry=false 2023-07-30 23:55:42,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43645,1690761333818 on jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43645,1690761333818, splitWal=true, meta=false 2023-07-30 23:55:42,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,43645,1690761333818 (carryingMeta=false) jenkins-hbase4.apache.org,43645,1690761333818/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@11e94264[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:55:42,313 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,37453,1690761333752 already deleted, retry=false 2023-07-30 23:55:42,313 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,37453,1690761333752 on jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,314 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,37453,1690761333752, splitWal=true, meta=false 2023-07-30 23:55:42,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,37453,1690761333752 (carryingMeta=false) jenkins-hbase4.apache.org,37453,1690761333752/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@55601d75[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:55:42,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,35865,1690761333726 already deleted, retry=false 2023-07-30 23:55:42,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,35865,1690761333726 on jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,35865,1690761333726, splitWal=true, meta=false 2023-07-30 23:55:42,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,35865,1690761333726 (carryingMeta=false) jenkins-hbase4.apache.org,35865,1690761333726/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@e419fdf[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:55:42,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43687,1690761333781 already deleted, retry=false 2023-07-30 23:55:42,318 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43687,1690761333781 on jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43687,1690761333781, splitWal=true, meta=true 2023-07-30 23:55:42,318 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,43687,1690761333781 (carryingMeta=true) jenkins-hbase4.apache.org,43687,1690761333781/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@73a1777d[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:55:42,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-30 23:55:42,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-30 23:55:42,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-30 23:55:42,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-30 23:55:42,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-30 23:55:42,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-30 23:55:42,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-30 23:55:42,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-30 23:55:42,334 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:42,334 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:55:42,334 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:55:42,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37557,1690761342123, sessionid=0x101b8a8e9d9001c, setting cluster-up flag (Was=false) 2023-07-30 23:55:42,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-30 23:55:42,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,342 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-30 23:55:42,343 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:55:42,343 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.hbase-snapshot/.tmp 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:42,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690761372347 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-30 23:55:42,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-30 23:55:42,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-30 23:55:42,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-30 23:55:42,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-30 23:55:42,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-30 23:55:42,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-30 23:55:42,351 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37453,1690761333752; numProcessing=1 2023-07-30 23:55:42,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761342351,5,FailOnTimeoutGroup] 2023-07-30 23:55:42,351 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43687,1690761333781; numProcessing=2 2023-07-30 23:55:42,351 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43645,1690761333818; numProcessing=3 2023-07-30 23:55:42,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761342351,5,FailOnTimeoutGroup] 2023-07-30 23:55:42,352 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35865,1690761333726; numProcessing=4 2023-07-30 23:55:42,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,352 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,35865,1690761333726, splitWal=true, meta=false 2023-07-30 23:55:42,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-30 23:55:42,352 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43645,1690761333818, splitWal=true, meta=false 2023-07-30 23:55:42,352 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43687,1690761333781, splitWal=true, meta=true 2023-07-30 23:55:42,352 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,37453,1690761333752, splitWal=true, meta=false 2023-07-30 23:55:42,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690761342352, completionTime=-1 2023-07-30 23:55:42,352 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-30 23:55:42,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-30 23:55:42,354 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43687,1690761333781, splitWal=true, meta=true, isMeta: true 2023-07-30 23:55:42,356 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781-splitting 2023-07-30 23:55:42,357 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781-splitting dir is empty, no logs to split. 2023-07-30 23:55:42,357 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,43687,1690761333781 WAL count=0, meta=true 2023-07-30 23:55:42,360 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781-splitting dir is empty, no logs to split. 2023-07-30 23:55:42,360 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,43687,1690761333781 WAL count=0, meta=true 2023-07-30 23:55:42,360 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43687,1690761333781 WAL splitting is done? wals=0, meta=true 2023-07-30 23:55:42,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-30 23:55:42,362 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-30 23:55:42,364 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:55:42,366 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(951): ClusterId : 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:42,366 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:55:42,368 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:55:42,368 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:55:42,370 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:55:42,371 DEBUG [RS:0;jenkins-hbase4:46759] zookeeper.ReadOnlyZKClient(139): Connect 0x016fe3ec to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:42,376 DEBUG [RS:0;jenkins-hbase4:46759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@fa80db1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:42,376 DEBUG [RS:0;jenkins-hbase4:46759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b97889c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:55:42,386 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46759 2023-07-30 23:55:42,386 INFO [RS:0;jenkins-hbase4:46759] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:55:42,386 INFO [RS:0;jenkins-hbase4:46759] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:55:42,386 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:55:42,386 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37557,1690761342123 with isa=jenkins-hbase4.apache.org/172.31.14.131:46759, startcode=1690761342152 2023-07-30 23:55:42,387 DEBUG [RS:0;jenkins-hbase4:46759] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:55:42,388 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60915, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:55:42,389 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37557] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,390 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:55:42,390 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35081 2023-07-30 23:55:42,390 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:55:42,394 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:55:42,394 DEBUG [RS:0;jenkins-hbase4:46759] zookeeper.ZKUtil(162): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,395 WARN [RS:0;jenkins-hbase4:46759] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:55:42,395 INFO [RS:0;jenkins-hbase4:46759] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:42,395 INFO [RS:0;jenkins-hbase4:46759] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:42,395 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,395 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46759,1690761342152] 2023-07-30 23:55:42,399 DEBUG [RS:0;jenkins-hbase4:46759] zookeeper.ZKUtil(162): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,400 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:42,400 INFO [RS:0;jenkins-hbase4:46759] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:55:42,402 INFO [RS:0;jenkins-hbase4:46759] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:55:42,402 INFO [RS:0;jenkins-hbase4:46759] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:55:42,402 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,402 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:55:42,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=51ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-30 23:55:42,403 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,403 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,404 DEBUG [RS:0;jenkins-hbase4:46759] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:55:42,405 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,405 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,405 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,416 INFO [RS:0;jenkins-hbase4:46759] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:55:42,416 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46759,1690761342152-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:42,426 INFO [RS:0;jenkins-hbase4:46759] regionserver.Replication(203): jenkins-hbase4.apache.org,46759,1690761342152 started 2023-07-30 23:55:42,426 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46759,1690761342152, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46759, sessionid=0x101b8a8e9d9001d 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46759,1690761342152' 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46759,1690761342152' 2023-07-30 23:55:42,427 DEBUG [RS:0;jenkins-hbase4:46759] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-30 23:55:42,428 DEBUG [RS:0;jenkins-hbase4:46759] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-30 23:55:42,428 DEBUG [RS:0;jenkins-hbase4:46759] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:55:42,428 INFO [RS:0;jenkins-hbase4:46759] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:55:42,428 INFO [RS:0;jenkins-hbase4:46759] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:55:42,514 DEBUG [jenkins-hbase4:37557] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-30 23:55:42,515 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46759,1690761342152, state=OPENING 2023-07-30 23:55:42,518 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-30 23:55:42,518 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-30 23:55:42,518 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46759,1690761342152}] 2023-07-30 23:55:42,528 INFO [RS:0;jenkins-hbase4:46759] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:42,530 INFO [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46759%2C1690761342152.jenkins-hbase4.apache.org%2C46759%2C1690761342152.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:42,549 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:42,549 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:42,549 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:42,551 INFO [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152/jenkins-hbase4.apache.org%2C46759%2C1690761342152.jenkins-hbase4.apache.org%2C46759%2C1690761342152.regiongroup-0.1690761342531 2023-07-30 23:55:42,551 DEBUG [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:42,672 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:42,672 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:42,673 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52004, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:42,677 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-30 23:55:42,677 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:42,677 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:42,678 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:42,679 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46759%2C1690761342152.meta, suffix=.meta, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:42,694 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:42,694 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:42,694 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:55:42,696 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152/jenkins-hbase4.apache.org%2C46759%2C1690761342152.meta.1690761342680.meta 2023-07-30 23:55:42,697 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK]] 2023-07-30 23:55:42,697 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:42,697 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-30 23:55:42,697 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-30 23:55:42,698 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-30 23:55:42,699 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-30 23:55:42,699 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:42,699 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-30 23:55:42,699 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-30 23:55:42,702 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:55:42,703 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:55:42,703 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:55:42,703 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:55:42,709 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/11fef2fd43a54b52a3a56fd6edead3d8 2023-07-30 23:55:42,710 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:42,710 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:55:42,710 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:42,711 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:55:42,711 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:55:42,711 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:42,712 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:55:42,712 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:55:42,712 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:55:42,713 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:55:42,718 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table/1f119aa9b99642ff9ae26cb85132b1b4 2023-07-30 23:55:42,719 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:42,719 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:55:42,720 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:55:42,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:55:42,724 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:55:42,725 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73419966, jitterRate=0.09404274821281433}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:55:42,725 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:55:42,725 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1690761342672 2023-07-30 23:55:42,730 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-30 23:55:42,731 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-30 23:55:42,731 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46759,1690761342152, state=OPEN 2023-07-30 23:55:42,732 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-30 23:55:42,733 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-30 23:55:42,735 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-30 23:55:42,735 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46759,1690761342152 in 215 msec 2023-07-30 23:55:42,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-07-30 23:55:42,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 374 msec 2023-07-30 23:55:43,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-30 23:55:45,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-30 23:55:46,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-30 23:55:46,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-30 23:55:46,864 DEBUG [hconnection-0x26eff476-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:46,865 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52012, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:46,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=51a075540234ea8884d31403732b8710, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43687,1690761333781, regionLocation=jenkins-hbase4.apache.org,43687,1690761333781, openSeqNum=2 2023-07-30 23:55:46,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6494f975164328a819a48f1beabe91f9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35865,1690761333726, regionLocation=jenkins-hbase4.apache.org,35865,1690761333726, openSeqNum=2 2023-07-30 23:55:46,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43687,1690761333781, regionLocation=jenkins-hbase4.apache.org,43687,1690761333781, openSeqNum=2 2023-07-30 23:55:46,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-30 23:55:46,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690761406875 2023-07-30 23:55:46,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690761466875 2023-07-30 23:55:46,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-07-30 23:55:46,889 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:55:46,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37557,1690761342123-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:46,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37557,1690761342123-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:46,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37557,1690761342123-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:46,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37557, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:46,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-30 23:55:46,898 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43645,1690761333818 had 0 regions 2023-07-30 23:55:46,899 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. is NOT online; state={51a075540234ea8884d31403732b8710 state=OPEN, ts=1690761346875, server=jenkins-hbase4.apache.org,43687,1690761333781}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-30 23:55:46,899 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43687,1690761333781 had 3 regions 2023-07-30 23:55:46,899 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,35865,1690761333726 had 1 regions 2023-07-30 23:55:46,899 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,37453,1690761333752 had 0 regions 2023-07-30 23:55:46,900 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43645,1690761333818, splitWal=true, meta=false, isMeta: false 2023-07-30 23:55:46,901 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,35865,1690761333726, splitWal=true, meta=false, isMeta: false 2023-07-30 23:55:46,901 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,37453,1690761333752, splitWal=true, meta=false, isMeta: false 2023-07-30 23:55:46,902 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43687,1690761333781, splitWal=true, meta=true, isMeta: false 2023-07-30 23:55:46,903 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818-splitting 2023-07-30 23:55:46,904 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,904 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,43645,1690761333818 WAL count=0, meta=false 2023-07-30 23:55:46,904 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726-splitting 2023-07-30 23:55:46,908 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,908 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,35865,1690761333726 WAL count=0, meta=false 2023-07-30 23:55:46,910 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752-splitting 2023-07-30 23:55:46,910 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,43687,1690761333781/hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., unknown_server=jenkins-hbase4.apache.org,35865,1690761333726/t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9., unknown_server=jenkins-hbase4.apache.org,43687,1690761333781/t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:46,911 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,911 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,37453,1690761333752 WAL count=0, meta=false 2023-07-30 23:55:46,912 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,912 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,43687,1690761333781 WAL count=0, meta=false 2023-07-30 23:55:46,913 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,913 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,43645,1690761333818 WAL count=0, meta=false 2023-07-30 23:55:46,913 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43645,1690761333818 WAL splitting is done? wals=0, meta=false 2023-07-30 23:55:46,914 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,914 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,35865,1690761333726 WAL count=0, meta=false 2023-07-30 23:55:46,914 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,35865,1690761333726 WAL splitting is done? wals=0, meta=false 2023-07-30 23:55:46,915 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,915 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,37453,1690761333752 WAL count=0, meta=false 2023-07-30 23:55:46,915 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,37453,1690761333752 WAL splitting is done? wals=0, meta=false 2023-07-30 23:55:46,916 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43687,1690761333781-splitting dir is empty, no logs to split. 2023-07-30 23:55:46,916 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,43687,1690761333781 WAL count=0, meta=false 2023-07-30 23:55:46,916 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43687,1690761333781 WAL splitting is done? wals=0, meta=false 2023-07-30 23:55:46,916 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43645,1690761333818 failed, ignore...File hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,43645,1690761333818-splitting does not exist. 2023-07-30 23:55:46,917 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,35865,1690761333726 failed, ignore...File hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,35865,1690761333726-splitting does not exist. 2023-07-30 23:55:46,917 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,37453,1690761333752 failed, ignore...File hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,37453,1690761333752-splitting does not exist. 2023-07-30 23:55:46,918 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN}, {pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN}] 2023-07-30 23:55:46,918 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43645,1690761333818 after splitting done 2023-07-30 23:55:46,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN}] 2023-07-30 23:55:46,919 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43645,1690761333818 from processing; numProcessing=3 2023-07-30 23:55:46,919 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,37453,1690761333752 after splitting done 2023-07-30 23:55:46,919 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,37453,1690761333752 from processing; numProcessing=2 2023-07-30 23:55:46,920 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN 2023-07-30 23:55:46,920 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN 2023-07-30 23:55:46,920 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN 2023-07-30 23:55:46,921 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:55:46,921 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:55:46,921 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43645,1690761333818, splitWal=true, meta=false in 4.6130 sec 2023-07-30 23:55:46,922 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:55:46,922 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,37453,1690761333752, splitWal=true, meta=false in 4.6060 sec 2023-07-30 23:55:47,071 DEBUG [jenkins-hbase4:37557] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-30 23:55:47,074 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=51a075540234ea8884d31403732b8710, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:47,074 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:47,074 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=6494f975164328a819a48f1beabe91f9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:47,074 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761347074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761347074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761347074"}]},"ts":"1690761347074"} 2023-07-30 23:55:47,074 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761347074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761347074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761347074"}]},"ts":"1690761347074"} 2023-07-30 23:55:47,074 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761347074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761347074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761347074"}]},"ts":"1690761347074"} 2023-07-30 23:55:47,077 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure 51a075540234ea8884d31403732b8710, server=jenkins-hbase4.apache.org,46759,1690761342152}] 2023-07-30 23:55:47,078 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=23, state=RUNNABLE; OpenRegionProcedure 6494f975164328a819a48f1beabe91f9, server=jenkins-hbase4.apache.org,46759,1690761342152}] 2023-07-30 23:55:47,079 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=22, state=RUNNABLE; OpenRegionProcedure ee738a1d08d169b10fce15cc4dc0d043, server=jenkins-hbase4.apache.org,46759,1690761342152}] 2023-07-30 23:55:47,233 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:47,233 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:55:47,235 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46759%2C1690761342152.jenkins-hbase4.apache.org%2C46759%2C1690761342152.regiongroup-1, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:55:47,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:47,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6494f975164328a819a48f1beabe91f9, NAME => 't1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:47,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:47,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,238 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,239 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1 2023-07-30 23:55:47,239 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1 2023-07-30 23:55:47,239 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6494f975164328a819a48f1beabe91f9 columnFamilyName cf1 2023-07-30 23:55:47,247 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/00173fc536fb4d1b8a36c88d95b68e51 2023-07-30 23:55:47,247 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(310): Store=6494f975164328a819a48f1beabe91f9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:47,247 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,248 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/norep 2023-07-30 23:55:47,248 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/norep 2023-07-30 23:55:47,248 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6494f975164328a819a48f1beabe91f9 columnFamilyName norep 2023-07-30 23:55:47,249 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(310): Store=6494f975164328a819a48f1beabe91f9/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:47,250 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:55:47,250 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:55:47,250 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:55:47,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:55:47,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6494f975164328a819a48f1beabe91f9; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58770846, jitterRate=-0.12424615025520325}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:55:47,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:55:47,260 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152/jenkins-hbase4.apache.org%2C46759%2C1690761342152.jenkins-hbase4.apache.org%2C46759%2C1690761342152.regiongroup-1.1690761347235 2023-07-30 23:55:47,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK]] 2023-07-30 23:55:47,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 51a075540234ea8884d31403732b8710, NAME => 'hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:47,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9., pid=25, masterSystemTime=1690761347229 2023-07-30 23:55:47,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:47,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:47,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:55:47,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:47,264 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee738a1d08d169b10fce15cc4dc0d043, NAME => 't2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:55:47,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:55:47,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,264 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=6494f975164328a819a48f1beabe91f9, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:47,265 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761347264"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761347264"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761347264"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761347264"}]},"ts":"1690761347264"} 2023-07-30 23:55:47,265 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info 2023-07-30 23:55:47,265 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info 2023-07-30 23:55:47,265 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51a075540234ea8884d31403732b8710 columnFamilyName info 2023-07-30 23:55:47,266 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,267 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1 2023-07-30 23:55:47,267 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1 2023-07-30 23:55:47,267 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee738a1d08d169b10fce15cc4dc0d043 columnFamilyName cf1 2023-07-30 23:55:47,271 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=23 2023-07-30 23:55:47,272 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=23, state=SUCCESS; OpenRegionProcedure 6494f975164328a819a48f1beabe91f9, server=jenkins-hbase4.apache.org,46759,1690761342152 in 191 msec 2023-07-30 23:55:47,274 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=17 2023-07-30 23:55:47,274 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,35865,1690761333726 after splitting done 2023-07-30 23:55:47,274 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN in 354 msec 2023-07-30 23:55:47,274 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,35865,1690761333726 from processing; numProcessing=1 2023-07-30 23:55:47,276 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,35865,1690761333726, splitWal=true, meta=false in 4.9590 sec 2023-07-30 23:55:47,278 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/39f56f6009014b15a197bd88c2d5fadc 2023-07-30 23:55:47,278 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info/fba57fb05b8f46a0a780d0a7c0f6febf 2023-07-30 23:55:47,278 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(310): Store=51a075540234ea8884d31403732b8710/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:47,278 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(310): Store=ee738a1d08d169b10fce15cc4dc0d043/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:47,279 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,279 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,280 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/norep 2023-07-30 23:55:47,280 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/norep 2023-07-30 23:55:47,280 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee738a1d08d169b10fce15cc4dc0d043 columnFamilyName norep 2023-07-30 23:55:47,280 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,281 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(310): Store=ee738a1d08d169b10fce15cc4dc0d043/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:55:47,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 51a075540234ea8884d31403732b8710 2023-07-30 23:55:47,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:55:47,285 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 51a075540234ea8884d31403732b8710; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68393951, jitterRate=0.019149288535118103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:55:47,285 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:55:47,285 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., pid=24, masterSystemTime=1690761347229 2023-07-30 23:55:47,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:55:47,287 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:47,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee738a1d08d169b10fce15cc4dc0d043; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72304437, jitterRate=0.07742007076740265}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:55:47,287 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:55:47,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:55:47,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=51a075540234ea8884d31403732b8710, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:47,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761347287"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761347287"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761347287"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761347287"}]},"ts":"1690761347287"} 2023-07-30 23:55:47,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043., pid=26, masterSystemTime=1690761347229 2023-07-30 23:55:47,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:47,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:55:47,290 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:55:47,290 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761347290"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761347290"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761347290"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761347290"}]},"ts":"1690761347290"} 2023-07-30 23:55:47,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-30 23:55:47,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure 51a075540234ea8884d31403732b8710, server=jenkins-hbase4.apache.org,46759,1690761342152 in 212 msec 2023-07-30 23:55:47,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN in 373 msec 2023-07-30 23:55:47,294 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=22 2023-07-30 23:55:47,294 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=22, state=SUCCESS; OpenRegionProcedure ee738a1d08d169b10fce15cc4dc0d043, server=jenkins-hbase4.apache.org,46759,1690761342152 in 214 msec 2023-07-30 23:55:47,296 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=18 2023-07-30 23:55:47,296 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43687,1690761333781 after splitting done 2023-07-30 23:55:47,296 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN in 376 msec 2023-07-30 23:55:47,296 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43687,1690761333781 from processing; numProcessing=0 2023-07-30 23:55:47,297 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43687,1690761333781, splitWal=true, meta=true in 4.9780 sec 2023-07-30 23:55:47,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:47,900 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-30 23:55:47,914 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-30 23:55:47,919 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-30 23:55:47,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.745sec 2023-07-30 23:55:47,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-30 23:55:47,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-30 23:55:47,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-30 23:55:47,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37557,1690761342123-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-30 23:55:47,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37557,1690761342123-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-30 23:55:47,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-30 23:55:47,970 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x571250c9 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:47,976 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@bc12d17, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:47,977 DEBUG [hconnection-0x5a18c453-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:47,979 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52028, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:47,983 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-30 23:55:47,983 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x571250c9 to 127.0.0.1:52585 2023-07-30 23:55:47,983 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:55:47,984 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:37557 after: jenkins-hbase4.apache.org:37557 2023-07-30 23:55:47,984 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x6daa8928 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:47,989 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@718ce292, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:48,400 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-30 23:55:48,401 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-30 23:55:48,401 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-30 23:55:48,402 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-30 23:55:48,491 DEBUG [hconnection-0x5a798b5b-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:48,492 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52030, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:55:48,514 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1690761348514 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:48,518 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication16907613485140x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:48,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1690761348514-0x101b8a8e9d90022 connected Start Replication Server start 2023-07-30 23:55:48,521 INFO [Listener at localhost/42375] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:55:48,522 INFO [Listener at localhost/42375] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:55:48,534 INFO [Listener at localhost/42375] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-30 23:55:48,535 DEBUG [Listener at localhost/42375] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:55:48,545 INFO [Listener at localhost/42375] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:55:48,546 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,36237,1690761329770, jenkins-hbase4.apache.org,42313,1690761329724] other RSs: [] 2023-07-30 23:55:48,567 DEBUG [Listener at localhost/42375.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x7f4fbb9c to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:48,574 DEBUG [Listener at localhost/42375.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1183c429, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:48,575 INFO [Listener at localhost/42375.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:48,579 DEBUG [Listener at localhost/42375.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:48,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d90024 connected 2023-07-30 23:55:48,580 INFO [Listener at localhost/42375.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:52,152 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,36237,1690761329770/1's WALs to 1690761348521.syncuptool.replication.org,1234,1 2023-07-30 23:55:52,154 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 with data 2023-07-30 23:55:52,154 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817 with data PBUF\x08\xDFU 2023-07-30 23:55:52,157 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,36237,1690761329770/1's WALs to 1690761348521.syncuptool.replication.org,1234,1 2023-07-30 23:55:52,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:52,740 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:55:54,159 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36237,1690761329770] 2023-07-30 23:55:54,159 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36237,1690761329770] 2023-07-30 23:55:54,175 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36237,1690761329770] 2023-07-30 23:55:54,175 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,36237,1690761329770, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:55:54,175 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,196 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] zookeeper.ReadOnlyZKClient(139): Connect 0x1838b490 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:54,202 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@68072e17, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:54,202 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:54,206 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:54,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d90026 connected 2023-07-30 23:55:54,207 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,36237,1690761329770 (queues=2) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:54,207 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1 2023-07-30 23:55:54,211 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,36237,1690761329770, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:54,211 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0 2023-07-30 23:55:54,211 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1 2023-07-30 23:55:54,213 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,36237,1690761329770, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:54,213 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0 2023-07-30 23:55:54,218 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 2023-07-30 23:55:54,218 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-30 23:55:54,218 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,36237,1690761329770 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:54,219 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,221 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817 2023-07-30 23:55:54,221 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-30 23:55:54,221 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-30 23:55:54,222 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,36237,1690761329770 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/409ccc1344e052df313f0ebb0776f6ff/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-30 23:55:54,224 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:54,226 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47064, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:54,227 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46759] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:54,228 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46759] zookeeper.ReadOnlyZKClient(139): Connect 0x2dc21def to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:54,234 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2305e627, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:54,235 DEBUG [hconnection-0x273a769f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:55:54,237 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47078, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-30 23:55:54,246 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46759] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:54,246 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36237,1690761329770] 2023-07-30 23:55:54,248 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1.1690761335817] 2023-07-30 23:55:54,252 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,36237,1690761329770 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:54,252 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,319 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,321 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] zookeeper.ZKUtil(162): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 2023-07-30 23:55:54,323 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,36237,1690761329770/jenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0.1690761331137 2023-07-30 23:55:54,323 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,323 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,323 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-0,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,36237,1690761329770 with the following stats: Total replicated edits: 1, current progress: 2023-07-30 23:55:54,423 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,36237,1690761329770 2023-07-30 23:55:54,424 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] zookeeper.ZKUtil(243): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Unable to list children of znode /1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,36237,1690761329770 because node does not exist (not an error) 2023-07-30 23:55:54,424 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770.replicationSource.shipperjenkins-hbase4.apache.org%2C36237%2C1690761329770.jenkins-hbase4.apache.org%2C36237%2C1690761329770.regiongroup-1,1-jenkins-hbase4.apache.org,36237,1690761329770] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,36237,1690761329770 with the following stats: Total replicated edits: 1, current progress: 2023-07-30 23:55:56,483 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42313,1690761329724/1's WALs to 1690761348521.syncuptool.replication.org,1234,1 2023-07-30 23:55:56,485 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 with data PBUF\x08\xF8\xA9\x01 2023-07-30 23:55:56,486 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141 with data PBUF\x08\xCD\x06 2023-07-30 23:55:56,488 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,42313,1690761329724/1's WALs to 1690761348521.syncuptool.replication.org,1234,1 2023-07-30 23:55:57,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:55:57,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 1 2023-07-30 23:55:58,490 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:58,491 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:58,501 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:58,502 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,42313,1690761329724, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:55:58,502 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:58,526 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] zookeeper.ReadOnlyZKClient(139): Connect 0x25ef9405 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:55:58,530 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7d8ec960, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:55:58,531 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:55:58,534 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:55:58,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d90029 connected 2023-07-30 23:55:58,536 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,42313,1690761329724 (queues=2) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:55:58,536 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0 2023-07-30 23:55:58,537 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42313,1690761329724, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:58,537 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1 2023-07-30 23:55:58,537 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0 2023-07-30 23:55:58,538 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42313,1690761329724, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:55:58,539 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1 2023-07-30 23:55:58,546 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137 2023-07-30 23:55:58,546 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-30 23:55:58,546 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-30 23:55:58,548 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42313,1690761329724 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-30 23:55:58,551 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:55:58,553 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47082, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:55:58,554 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141 2023-07-30 23:55:58,554 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:55:58,554 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-30 23:55:58,554 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46759] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:55:58,554 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42313,1690761329724 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-07-30 23:55:58,555 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:58,558 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1.1690761332141] 2023-07-30 23:55:58,559 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42313,1690761329724 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:58,559 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:58,570 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46759] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:55:58,571 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42313,1690761329724] 2023-07-30 23:55:58,572 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0.1690761331137] 2023-07-30 23:55:58,574 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42313,1690761329724 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:55:58,574 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:58,659 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:58,663 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:58,663 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-1,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42313,1690761329724 with the following stats: Total replicated edits: 1, current progress: 2023-07-30 23:55:58,764 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42313,1690761329724 2023-07-30 23:55:58,767 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] zookeeper.ZKUtil(243): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Unable to list children of znode /1/replication/rs/1690761348521.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42313,1690761329724 because node does not exist (not an error) 2023-07-30 23:55:58,768 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724.replicationSource.shipperjenkins-hbase4.apache.org%2C42313%2C1690761329724.jenkins-hbase4.apache.org%2C42313%2C1690761329724.regiongroup-0,1-jenkins-hbase4.apache.org,42313,1690761329724] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42313,1690761329724 with the following stats: Total replicated edits: 1, current progress: 2023-07-30 23:55:59,823 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:56:02,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:56:02,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-30 23:56:07,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:56:07,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-30 23:56:08,548 INFO [Listener at localhost/42375] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-30 23:56:08,657 DEBUG [Listener at localhost/42375.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90024, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:08,657 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f4fbb9c to 127.0.0.1:52585 2023-07-30 23:56:08,658 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:08,861 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:08,861 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761348514-0x101b8a8e9d90022, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:08,874 INFO [Listener at localhost/42375] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-30 23:56:08,874 DEBUG [Listener at localhost/42375] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-30 23:56:08,876 INFO [Listener at localhost/42375] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-30 23:56:08,876 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:08,876 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:08,877 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:56:08,877 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:08,877 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:56:08,877 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:56:08,878 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45867 2023-07-30 23:56:08,879 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:08,880 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:08,882 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45867 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:08,890 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:458670x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:08,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45867-0x101b8a8e9d9002a connected 2023-07-30 23:56:08,896 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:56:08,897 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:56:08,897 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-30 23:56:08,903 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45867 2023-07-30 23:56:08,903 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45867 2023-07-30 23:56:08,906 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45867 2023-07-30 23:56:08,907 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45867 2023-07-30 23:56:08,908 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45867 2023-07-30 23:56:08,908 INFO [Listener at localhost/42375] master.HMaster(444): hbase.rootdir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5, hbase.cluster.distributed=false 2023-07-30 23:56:08,912 DEBUG [pool-560-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-30 23:56:08,938 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-30 23:56:08,938 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:08,938 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:08,938 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:56:08,938 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:08,938 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:56:08,939 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:56:08,948 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43785 2023-07-30 23:56:08,948 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:56:08,952 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:56:08,952 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:08,953 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:08,954 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43785 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:08,958 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:437850x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:08,960 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:437850x0, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:56:08,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43785-0x101b8a8e9d9002b connected 2023-07-30 23:56:08,961 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:56:08,962 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-30 23:56:08,966 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43785 2023-07-30 23:56:08,967 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43785 2023-07-30 23:56:08,968 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43785 2023-07-30 23:56:08,972 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43785 2023-07-30 23:56:08,975 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43785 2023-07-30 23:56:08,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:08,999 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-30 23:56:09,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,002 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-30 23:56:09,002 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-30 23:56:09,002 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:56:09,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-30 23:56:09,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-30 23:56:09,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,45867,1690761368875 from backup master directory 2023-07-30 23:56:09,008 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,008 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-30 23:56:09,008 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:56:09,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:09,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2eb892d5 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:09,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@39192af5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:09,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:56:09,087 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-30 23:56:09,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:09,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:09,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479 to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479-dead as it is dead 2023-07-30 23:56:09,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479-dead/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 2023-07-30 23:56:09,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479-dead/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 after 0ms 2023-07-30 23:56:09,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479-dead/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 to hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 2023-07-30 23:56:09,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,43527,1690761328479-dead 2023-07-30 23:56:09,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:09,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45867%2C1690761368875.jenkins-hbase4.apache.org%2C45867%2C1690761368875.regiongroup-0, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,45867,1690761368875, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/oldWALs, maxLogs=10 2023-07-30 23:56:09,119 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:56:09,119 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:56:09,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/WALs/jenkins-hbase4.apache.org,45867,1690761368875/jenkins-hbase4.apache.org%2C45867%2C1690761368875.jenkins-hbase4.apache.org%2C45867%2C1690761368875.regiongroup-0.1690761369102 2023-07-30 23:56:09,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK], DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK]] 2023-07-30 23:56:09,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:09,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:09,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:09,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:09,129 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:09,130 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-30 23:56:09,130 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-30 23:56:09,139 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c20b0c3094344a888ac1fd92ef271752 2023-07-30 23:56:09,139 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:09,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-30 23:56:09,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 2023-07-30 23:56:09,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 2023-07-30 23:56:09,148 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43527%2C1690761328479.jenkins-hbase4.apache.org%2C43527%2C1690761328479.regiongroup-0.1690761330195 2023-07-30 23:56:09,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:09,154 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-30 23:56:09,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71760091, jitterRate=0.06930868327617645}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:56:09,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:56:09,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-30 23:56:09,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-30 23:56:09,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-30 23:56:09,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-30 23:56:09,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-30 23:56:09,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-30 23:56:09,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-30 23:56:09,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-30 23:56:09,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-30 23:56:09,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-30 23:56:09,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-30 23:56:09,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-30 23:56:09,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-30 23:56:09,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-30 23:56:09,167 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-30 23:56:09,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42313,1690761329724, table=hbase:meta, region=1588230740 2023-07-30 23:56:09,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-30 23:56:09,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,36237,1690761329770 already deleted, retry=false 2023-07-30 23:56:09,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,36237,1690761329770 on jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,36237,1690761329770, splitWal=true, meta=false 2023-07-30 23:56:09,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,36237,1690761329770 (carryingMeta=false) jenkins-hbase4.apache.org,36237,1690761329770/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@67420006[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:56:09,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42313,1690761329724 already deleted, retry=false 2023-07-30 23:56:09,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42313,1690761329724 on jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,175 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42313,1690761329724, splitWal=true, meta=true 2023-07-30 23:56:09,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,42313,1690761329724 (carryingMeta=true) jenkins-hbase4.apache.org,42313,1690761329724/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@736276b9[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:56:09,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-30 23:56:09,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-30 23:56:09,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-30 23:56:09,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-30 23:56:09,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-30 23:56:09,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-30 23:56:09,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-30 23:56:09,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-30 23:56:09,202 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-30 23:56:09,202 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-30 23:56:09,202 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:56:09,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45867,1690761368875, sessionid=0x101b8a8e9d9002a, setting cluster-up flag (Was=false) 2023-07-30 23:56:09,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-30 23:56:09,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-30 23:56:09,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:09,215 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/.hbase-snapshot/.tmp 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:56:09,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,230 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690761399230 2023-07-30 23:56:09,231 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-30 23:56:09,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-30 23:56:09,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-30 23:56:09,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-30 23:56:09,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-30 23:56:09,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-30 23:56:09,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,239 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42313,1690761329724; numProcessing=1 2023-07-30 23:56:09,240 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36237,1690761329770; numProcessing=2 2023-07-30 23:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-30 23:56:09,240 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36237,1690761329770, splitWal=true, meta=false 2023-07-30 23:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-30 23:56:09,240 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42313,1690761329724, splitWal=true, meta=true 2023-07-30 23:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-30 23:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-30 23:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-30 23:56:09,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761369240,5,FailOnTimeoutGroup] 2023-07-30 23:56:09,241 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42313,1690761329724, splitWal=true, meta=true, isMeta: true 2023-07-30 23:56:09,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761369241,5,FailOnTimeoutGroup] 2023-07-30 23:56:09,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-30 23:56:09,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690761369245, completionTime=-1 2023-07-30 23:56:09,246 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-30 23:56:09,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-30 23:56:09,247 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724-splitting 2023-07-30 23:56:09,247 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724-splitting dir is empty, no logs to split. 2023-07-30 23:56:09,248 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42313,1690761329724 WAL count=0, meta=true 2023-07-30 23:56:09,250 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724-splitting dir is empty, no logs to split. 2023-07-30 23:56:09,250 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42313,1690761329724 WAL count=0, meta=true 2023-07-30 23:56:09,250 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42313,1690761329724 WAL splitting is done? wals=0, meta=true 2023-07-30 23:56:09,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-30 23:56:09,260 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-30 23:56:09,262 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:09,277 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(951): ClusterId : c047cc16-085a-45ee-a021-aa0b6e22ca94 2023-07-30 23:56:09,277 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:56:09,279 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:56:09,279 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:56:09,282 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:56:09,284 DEBUG [RS:0;jenkins-hbase4:43785] zookeeper.ReadOnlyZKClient(139): Connect 0x0b0c92b1 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:09,289 DEBUG [RS:0;jenkins-hbase4:43785] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5cb7da0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:09,290 DEBUG [RS:0;jenkins-hbase4:43785] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f4f9c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:56:09,298 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43785 2023-07-30 23:56:09,298 INFO [RS:0;jenkins-hbase4:43785] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:56:09,298 INFO [RS:0;jenkins-hbase4:43785] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:56:09,298 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:56:09,299 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45867,1690761368875 with isa=jenkins-hbase4.apache.org/172.31.14.131:43785, startcode=1690761368937 2023-07-30 23:56:09,299 DEBUG [RS:0;jenkins-hbase4:43785] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:56:09,301 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50085, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:56:09,302 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45867] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,302 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5 2023-07-30 23:56:09,302 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45087 2023-07-30 23:56:09,302 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:56:09,303 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:56:09,304 DEBUG [RS:0;jenkins-hbase4:43785] zookeeper.ZKUtil(162): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,304 WARN [RS:0;jenkins-hbase4:43785] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:56:09,305 INFO [RS:0;jenkins-hbase4:43785] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:09,305 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:09,305 INFO [RS:0;jenkins-hbase4:43785] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:09,305 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,330 INFO [RS:0;jenkins-hbase4:43785] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-30 23:56:09,331 DEBUG [RS:0;jenkins-hbase4:43785] zookeeper.ZKUtil(162): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,332 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:56:09,332 INFO [RS:0;jenkins-hbase4:43785] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:56:09,338 INFO [RS:0;jenkins-hbase4:43785] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:56:09,342 INFO [RS:0;jenkins-hbase4:43785] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:56:09,342 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,343 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:56:09,345 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,345 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,345 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,345 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 DEBUG [RS:0;jenkins-hbase4:43785] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:09,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-30 23:56:09,350 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,351 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,351 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,364 INFO [RS:0;jenkins-hbase4:43785] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:56:09,365 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43785,1690761368937-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:09,376 INFO [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:56:09,378 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1690761348521.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:09,391 INFO [RS:0;jenkins-hbase4:43785] regionserver.Replication(203): jenkins-hbase4.apache.org,43785,1690761368937 started 2023-07-30 23:56:09,391 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43785,1690761368937, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43785, sessionid=0x101b8a8e9d9002b 2023-07-30 23:56:09,392 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:56:09,392 DEBUG [RS:0;jenkins-hbase4:43785] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,392 DEBUG [RS:0;jenkins-hbase4:43785] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43785,1690761368937' 2023-07-30 23:56:09,392 DEBUG [RS:0;jenkins-hbase4:43785] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-30 23:56:09,392 DEBUG [RS:0;jenkins-hbase4:43785] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43785,1690761368937' 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-30 23:56:09,393 DEBUG [RS:0;jenkins-hbase4:43785] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:56:09,393 INFO [RS:0;jenkins-hbase4:43785] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:56:09,394 INFO [RS:0;jenkins-hbase4:43785] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:56:09,401 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x70bff179 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:09,405 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4d0d3161, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:09,406 INFO [RS:0;jenkins-hbase4:43785.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:09,412 DEBUG [jenkins-hbase4:45867] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-30 23:56:09,417 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43785,1690761368937, state=OPENING 2023-07-30 23:56:09,418 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:09,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d9002f connected 2023-07-30 23:56:09,422 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-30 23:56:09,422 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43785,1690761368937}] 2023-07-30 23:56:09,423 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-30 23:56:09,428 INFO [RS:0;jenkins-hbase4:43785.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:56:09,494 INFO [RS:0;jenkins-hbase4:43785] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:09,497 INFO [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:56:09,516 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:56:09,516 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:56:09,534 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0 for peer 1 2023-07-30 23:56:09,534 INFO [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497 2023-07-30 23:56:09,534 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0 2023-07-30 23:56:09,535 INFO [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:56:09,538 DEBUG [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:56:09,538 INFO [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0 2023-07-30 23:56:09,545 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:09,579 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:09,579 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:56:09,581 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:56:09,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-30 23:56:09,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:09,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:09,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:09,587 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43785%2C1690761368937.meta, suffix=.meta, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:56:09,601 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:56:09,602 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:56:09,610 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.meta.1690761369587.meta 2023-07-30 23:56:09,611 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK], DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK]] 2023-07-30 23:56:09,611 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:09,611 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-30 23:56:09,611 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-30 23:56:09,611 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-30 23:56:09,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-30 23:56:09,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:09,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-30 23:56:09,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-30 23:56:09,615 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:56:09,616 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info 2023-07-30 23:56:09,616 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info 2023-07-30 23:56:09,617 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:56:09,625 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info/4b3131ff2cbb4bf289d46c3f198185af 2023-07-30 23:56:09,625 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:09,625 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:56:09,626 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:56:09,626 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:56:09,627 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:56:09,637 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier/6e9a76720bdb4beb853377b05d4385f2 2023-07-30 23:56:09,637 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:09,637 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:56:09,638 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table 2023-07-30 23:56:09,638 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table 2023-07-30 23:56:09,639 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:56:09,645 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/table/d18717fd499f4adbb84d85844f86d4bc 2023-07-30 23:56:09,645 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:09,646 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740 2023-07-30 23:56:09,648 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740 2023-07-30 23:56:09,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:56:09,652 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:09,652 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:56:09,654 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67001425, jitterRate=-0.0016009658575057983}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:56:09,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:56:09,655 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1690761369579 2023-07-30 23:56:09,662 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-30 23:56:09,662 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-30 23:56:09,663 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43785,1690761368937, state=OPEN 2023-07-30 23:56:09,664 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-30 23:56:09,665 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-30 23:56:09,666 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-30 23:56:09,667 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43785,1690761368937 in 243 msec 2023-07-30 23:56:09,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-30 23:56:09,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 412 msec 2023-07-30 23:56:09,758 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:09,863 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:09,967 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,072 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,178 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,284 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,389 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,494 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,598 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,703 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,808 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:10,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1602ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-30 23:56:10,912 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,017 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,122 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,227 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,332 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,437 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,542 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,648 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,753 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,857 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:11,962 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,067 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,172 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,278 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-30 23:56:12,383 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:56:12,426 INFO [jenkins-hbase4:46759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-30 23:56:12,489 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,594 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,700 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,805 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:12,909 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,015 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,121 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,227 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,332 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,438 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,543 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,649 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-30 23:56:13,753 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-30 23:56:13,754 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,756 DEBUG [hconnection-0x2d80f2e5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:13,757 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46490, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:56:13,763 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ef4a62973dc7531db8626f8122ae55c2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42313,1690761329724, regionLocation=jenkins-hbase4.apache.org,42313,1690761329724, openSeqNum=2 2023-07-30 23:56:13,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=409ccc1344e052df313f0ebb0776f6ff, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36237,1690761329770, regionLocation=jenkins-hbase4.apache.org,36237,1690761329770, openSeqNum=2 2023-07-30 23:56:13,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=bffda5817d519b05008d136099bf920f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42313,1690761329724, regionLocation=jenkins-hbase4.apache.org,42313,1690761329724, openSeqNum=2 2023-07-30 23:56:13,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-30 23:56:13,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690761433764 2023-07-30 23:56:13,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690761493764 2023-07-30 23:56:13,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-30 23:56:13,783 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42313,1690761329724 had 3 regions 2023-07-30 23:56:13,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1690761368875-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:13,783 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,36237,1690761329770 had 1 regions 2023-07-30 23:56:13,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1690761368875-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:13,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1690761368875-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:13,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45867, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:13,786 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:13,786 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. is NOT online; state={ef4a62973dc7531db8626f8122ae55c2 state=OPEN, ts=1690761373764, server=jenkins-hbase4.apache.org,42313,1690761329724}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-30 23:56:13,787 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42313,1690761329724, splitWal=true, meta=true, isMeta: false 2023-07-30 23:56:13,787 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36237,1690761329770, splitWal=true, meta=false, isMeta: false 2023-07-30 23:56:13,789 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724-splitting dir is empty, no logs to split. 2023-07-30 23:56:13,790 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,42313,1690761329724 WAL count=0, meta=false 2023-07-30 23:56:13,790 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770-splitting 2023-07-30 23:56:13,790 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,42313,1690761329724/hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2., unknown_server=jenkins-hbase4.apache.org,36237,1690761329770/t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff., unknown_server=jenkins-hbase4.apache.org,42313,1690761329724/t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:13,791 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770-splitting dir is empty, no logs to split. 2023-07-30 23:56:13,791 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,36237,1690761329770 WAL count=0, meta=false 2023-07-30 23:56:13,792 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,42313,1690761329724-splitting dir is empty, no logs to split. 2023-07-30 23:56:13,793 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,42313,1690761329724 WAL count=0, meta=false 2023-07-30 23:56:13,793 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42313,1690761329724 WAL splitting is done? wals=0, meta=false 2023-07-30 23:56:13,794 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770-splitting dir is empty, no logs to split. 2023-07-30 23:56:13,794 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,36237,1690761329770 WAL count=0, meta=false 2023-07-30 23:56:13,794 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36237,1690761329770 WAL splitting is done? wals=0, meta=false 2023-07-30 23:56:13,795 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN}, {pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN}] 2023-07-30 23:56:13,796 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN 2023-07-30 23:56:13,796 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN 2023-07-30 23:56:13,798 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,36237,1690761329770 failed, ignore...File hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,36237,1690761329770-splitting does not exist. 2023-07-30 23:56:13,799 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:13,800 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:13,800 DEBUG [jenkins-hbase4:45867] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-30 23:56:13,801 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN}] 2023-07-30 23:56:13,805 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=bffda5817d519b05008d136099bf920f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:13,805 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=ef4a62973dc7531db8626f8122ae55c2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:13,805 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761373805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761373805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761373805"}]},"ts":"1690761373805"} 2023-07-30 23:56:13,805 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761373805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761373805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761373805"}]},"ts":"1690761373805"} 2023-07-30 23:56:13,806 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN 2023-07-30 23:56:13,809 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:13,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure ef4a62973dc7531db8626f8122ae55c2, server=jenkins-hbase4.apache.org,43785,1690761368937}] 2023-07-30 23:56:13,809 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure bffda5817d519b05008d136099bf920f, server=jenkins-hbase4.apache.org,43785,1690761368937}] 2023-07-30 23:56:13,860 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,961 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=409ccc1344e052df313f0ebb0776f6ff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:13,961 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761373961"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761373961"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761373961"}]},"ts":"1690761373961"} 2023-07-30 23:56:13,964 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure 409ccc1344e052df313f0ebb0776f6ff, server=jenkins-hbase4.apache.org,43785,1690761368937}] 2023-07-30 23:56:13,969 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:13,969 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:13,971 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:13,972 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1, suffix=, logDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937, archiveDir=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs, maxLogs=10 2023-07-30 23:56:13,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:13,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bffda5817d519b05008d136099bf920f, NAME => 't2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:13,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup bffda5817d519b05008d136099bf920f 2023-07-30 23:56:13,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:13,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bffda5817d519b05008d136099bf920f 2023-07-30 23:56:13,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bffda5817d519b05008d136099bf920f 2023-07-30 23:56:13,977 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bffda5817d519b05008d136099bf920f 2023-07-30 23:56:13,979 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1 2023-07-30 23:56:13,979 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1 2023-07-30 23:56:13,979 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bffda5817d519b05008d136099bf920f columnFamilyName cf1 2023-07-30 23:56:13,989 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK] 2023-07-30 23:56:13,989 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK] 2023-07-30 23:56:13,989 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f91a173b57f14bb9b2758966745afc1d 2023-07-30 23:56:13,989 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1/f91a173b57f14bb9b2758966745afc1d 2023-07-30 23:56:13,989 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(310): Store=bffda5817d519b05008d136099bf920f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:13,990 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region bffda5817d519b05008d136099bf920f 2023-07-30 23:56:13,991 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep 2023-07-30 23:56:13,991 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep 2023-07-30 23:56:13,991 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bffda5817d519b05008d136099bf920f columnFamilyName norep 2023-07-30 23:56:14,004 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1 for peer 1 2023-07-30 23:56:14,004 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973 2023-07-30 23:56:14,005 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1 2023-07-30 23:56:14,005 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:56:14,006 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1 2023-07-30 23:56:14,006 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37233,DS-83f13fd2-a0ef-4e71-9abb-cf8194293a3f,DISK], DatanodeInfoWithStorage[127.0.0.1:42315,DS-b1f56040-4fdd-421b-b0c7-1e01d5c43cde,DISK]] 2023-07-30 23:56:14,006 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ef4a62973dc7531db8626f8122ae55c2, NAME => 'hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:14,006 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 867abfcf8f67400aba2b3ee4fd343393 2023-07-30 23:56:14,006 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,007 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:14,007 DEBUG [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep/867abfcf8f67400aba2b3ee4fd343393 2023-07-30 23:56:14,007 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,007 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,007 INFO [StoreOpener-bffda5817d519b05008d136099bf920f-1] regionserver.HStore(310): Store=bffda5817d519b05008d136099bf920f/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:14,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f 2023-07-30 23:56:14,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f 2023-07-30 23:56:14,010 INFO [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,011 DEBUG [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info 2023-07-30 23:56:14,011 DEBUG [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info 2023-07-30 23:56:14,011 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-30 23:56:14,012 INFO [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ef4a62973dc7531db8626f8122ae55c2 columnFamilyName info 2023-07-30 23:56:14,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:56:14,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bffda5817d519b05008d136099bf920f 2023-07-30 23:56:14,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bffda5817d519b05008d136099bf920f; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67345634, jitterRate=0.0035281479358673096}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:56:14,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bffda5817d519b05008d136099bf920f: 2023-07-30 23:56:14,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f., pid=26, masterSystemTime=1690761373962 2023-07-30 23:56:14,020 DEBUG [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/info/d7b0678a304d4276ba635aae66502e80 2023-07-30 23:56:14,020 INFO [StoreOpener-ef4a62973dc7531db8626f8122ae55c2-1] regionserver.HStore(310): Store=ef4a62973dc7531db8626f8122ae55c2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:14,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:14,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:14,020 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=bffda5817d519b05008d136099bf920f, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:14,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761374020"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761374020"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761374020"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761374020"}]},"ts":"1690761374020"} 2023-07-30 23:56:14,021 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,023 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-30 23:56:14,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure bffda5817d519b05008d136099bf920f, server=jenkins-hbase4.apache.org,43785,1690761368937 in 213 msec 2023-07-30 23:56:14,025 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=bffda5817d519b05008d136099bf920f, ASSIGN in 229 msec 2023-07-30 23:56:14,027 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:14,035 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ef4a62973dc7531db8626f8122ae55c2; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72059841, jitterRate=0.07377530634403229}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:56:14,035 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ef4a62973dc7531db8626f8122ae55c2: 2023-07-30 23:56:14,037 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2., pid=25, masterSystemTime=1690761373962 2023-07-30 23:56:14,039 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:14,039 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:14,039 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=ef4a62973dc7531db8626f8122ae55c2, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:14,040 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761374039"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761374039"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761374039"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761374039"}]},"ts":"1690761374039"} 2023-07-30 23:56:14,043 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-30 23:56:14,043 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure ef4a62973dc7531db8626f8122ae55c2, server=jenkins-hbase4.apache.org,43785,1690761368937 in 232 msec 2023-07-30 23:56:14,045 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=19 2023-07-30 23:56:14,045 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42313,1690761329724 after splitting done 2023-07-30 23:56:14,045 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ef4a62973dc7531db8626f8122ae55c2, ASSIGN in 248 msec 2023-07-30 23:56:14,045 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42313,1690761329724 from processing; numProcessing=1 2023-07-30 23:56:14,046 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42313,1690761329724, splitWal=true, meta=true in 4.8710 sec 2023-07-30 23:56:14,113 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,113 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:56:14,113 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,113 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497, lastWalPosition=454, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:56:14,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:14,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 409ccc1344e052df313f0ebb0776f6ff, NAME => 't1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:14,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:14,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,132 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,133 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1 2023-07-30 23:56:14,133 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1 2023-07-30 23:56:14,134 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 409ccc1344e052df313f0ebb0776f6ff columnFamilyName cf1 2023-07-30 23:56:14,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-30 23:56:14,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:56:14,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-30 23:56:14,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:56:14,143 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a3a52480850d45169a167bf2c35045a6 2023-07-30 23:56:14,143 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1/a3a52480850d45169a167bf2c35045a6 2023-07-30 23:56:14,143 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(310): Store=409ccc1344e052df313f0ebb0776f6ff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:14,143 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,144 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep 2023-07-30 23:56:14,145 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep 2023-07-30 23:56:14,145 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 409ccc1344e052df313f0ebb0776f6ff columnFamilyName norep 2023-07-30 23:56:14,151 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d50180c3272d4b76b6cde45b0d81dbbc 2023-07-30 23:56:14,151 DEBUG [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(539): loaded hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep/d50180c3272d4b76b6cde45b0d81dbbc 2023-07-30 23:56:14,152 INFO [StoreOpener-409ccc1344e052df313f0ebb0776f6ff-1] regionserver.HStore(310): Store=409ccc1344e052df313f0ebb0776f6ff/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:56:14,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:14,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 409ccc1344e052df313f0ebb0776f6ff; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73724783, jitterRate=0.09858487546443939}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:56:14,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 409ccc1344e052df313f0ebb0776f6ff: 2023-07-30 23:56:14,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff., pid=27, masterSystemTime=1690761374119 2023-07-30 23:56:14,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:14,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:14,163 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=409ccc1344e052df313f0ebb0776f6ff, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:14,163 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761374163"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761374163"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761374163"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761374163"}]},"ts":"1690761374163"} 2023-07-30 23:56:14,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-07-30 23:56:14,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure 409ccc1344e052df313f0ebb0776f6ff, server=jenkins-hbase4.apache.org,43785,1690761368937 in 201 msec 2023-07-30 23:56:14,169 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=18 2023-07-30 23:56:14,169 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,36237,1690761329770 after splitting done 2023-07-30 23:56:14,169 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=409ccc1344e052df313f0ebb0776f6ff, ASSIGN in 366 msec 2023-07-30 23:56:14,169 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,36237,1690761329770 from processing; numProcessing=0 2023-07-30 23:56:14,172 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36237,1690761329770, splitWal=true, meta=false in 4.9980 sec 2023-07-30 23:56:14,223 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,239 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:56:14,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,242 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:56:14,242 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,242 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973, lastWalPosition=791, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:56:14,328 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,347 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,389 INFO [jenkins-hbase4:43785Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0]: currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497 at position: 454 walGroup [jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1]: currently replicating from: hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973 at position: 791 2023-07-30 23:56:14,434 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,452 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,539 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,557 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,643 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,661 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,748 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,765 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,786 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-30 23:56:14,798 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-30 23:56:14,801 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-30 23:56:14,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.793sec 2023-07-30 23:56:14,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-30 23:56:14,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-30 23:56:14,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-30 23:56:14,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1690761368875-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-30 23:56:14,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1690761368875-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-30 23:56:14,804 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-30 23:56:14,852 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,869 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:14,883 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x47074a73 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:14,887 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3576067b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:14,889 DEBUG [hconnection-0x40ef486b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:14,890 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46496, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:56:14,896 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-30 23:56:14,896 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x47074a73 to 127.0.0.1:52585 2023-07-30 23:56:14,896 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:14,897 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:45867 after: jenkins-hbase4.apache.org:45867 2023-07-30 23:56:14,898 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x7c775526 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:14,905 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4661f6d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:14,905 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6daa8928 to 127.0.0.1:52585 2023-07-30 23:56:14,905 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:14,906 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-30 23:56:14,906 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(257): Found active master hash=969351842, stopped=false 2023-07-30 23:56:14,906 INFO [Listener at localhost/42375] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:56:14,909 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:56:14,909 INFO [Listener at localhost/42375] procedure2.ProcedureExecutor(629): Stopping 2023-07-30 23:56:14,909 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:56:14,909 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:56:14,909 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68f3d9bf to 127.0.0.1:52585 2023-07-30 23:56:14,910 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:14,910 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46759,1690761342152' ***** 2023-07-30 23:56:14,910 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:56:14,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:56:14,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:56:14,912 INFO [RS:0;jenkins-hbase4:46759] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:56:14,912 INFO [RS:0;jenkins-hbase4:46759] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:56:14,912 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:56:14,912 INFO [RS:0;jenkins-hbase4:46759] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:56:14,912 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(3305): Received CLOSE for 51a075540234ea8884d31403732b8710 2023-07-30 23:56:14,912 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(3305): Received CLOSE for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:14,912 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(3305): Received CLOSE for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:14,913 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:56:14,913 DEBUG [RS:0;jenkins-hbase4:46759] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x016fe3ec to 127.0.0.1:52585 2023-07-30 23:56:14,913 DEBUG [RS:0;jenkins-hbase4:46759] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:14,913 INFO [RS:0;jenkins-hbase4:46759] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:56:14,913 INFO [RS:0;jenkins-hbase4:46759] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:56:14,913 INFO [RS:0;jenkins-hbase4:46759] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:56:14,913 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-30 23:56:14,913 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-30 23:56:14,913 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1478): Online Regions={51a075540234ea8884d31403732b8710=hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., ee738a1d08d169b10fce15cc4dc0d043=t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043., 1588230740=hbase:meta,,1.1588230740, 6494f975164328a819a48f1beabe91f9=t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.} 2023-07-30 23:56:14,913 DEBUG [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1504): Waiting on 1588230740, 51a075540234ea8884d31403732b8710, 6494f975164328a819a48f1beabe91f9, ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 51a075540234ea8884d31403732b8710, disabling compactions & flushes 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:56:14,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:14,915 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. after waiting 0 ms 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:14,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:56:14,916 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:56:14,916 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-30 23:56:14,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-30 23:56:14,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:14,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:56:14,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:14,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee738a1d08d169b10fce15cc4dc0d043, disabling compactions & flushes 2023-07-30 23:56:14,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:14,931 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:56:14,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:14,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. after waiting 0 ms 2023-07-30 23:56:14,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:14,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ee738a1d08d169b10fce15cc4dc0d043 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-30 23:56:14,951 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/info/c8dd0467bc824bef8ca79cd38a517f7e 2023-07-30 23:56:14,960 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:14,962 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/.tmp/cf1/fd7a2b174c4d4b03b4f3bfc58987fffd 2023-07-30 23:56:14,963 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/info/c8dd0467bc824bef8ca79cd38a517f7e as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/c8dd0467bc824bef8ca79cd38a517f7e 2023-07-30 23:56:14,967 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd7a2b174c4d4b03b4f3bfc58987fffd 2023-07-30 23:56:14,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/.tmp/cf1/fd7a2b174c4d4b03b4f3bfc58987fffd as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/fd7a2b174c4d4b03b4f3bfc58987fffd 2023-07-30 23:56:14,968 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/c8dd0467bc824bef8ca79cd38a517f7e, entries=24, sequenceid=31, filesize=7.4 K 2023-07-30 23:56:14,970 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 54ms, sequenceid=31, compaction requested=false 2023-07-30 23:56:14,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:15,016 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x76c6a82b to 127.0.0.1:52585 2023-07-30 23:56:15,017 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd7a2b174c4d4b03b4f3bfc58987fffd 2023-07-30 23:56:15,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/fd7a2b174c4d4b03b4f3bfc58987fffd, entries=100, sequenceid=15, filesize=7.9 K 2023-07-30 23:56:15,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for ee738a1d08d169b10fce15cc4dc0d043 in 86ms, sequenceid=15, compaction requested=false 2023-07-30 23:56:15,029 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-30 23:56:15,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-30 23:56:15,031 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:56:15,031 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:56:15,031 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-30 23:56:15,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-07-30 23:56:15,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:15,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:56:15,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:15,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6494f975164328a819a48f1beabe91f9, disabling compactions & flushes 2023-07-30 23:56:15,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:15,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:15,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. after waiting 0 ms 2023-07-30 23:56:15,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:15,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6494f975164328a819a48f1beabe91f9 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-30 23:56:15,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/.tmp/cf1/fb344aa3832f4680b9d1d96be0c3aa77 2023-07-30 23:56:15,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb344aa3832f4680b9d1d96be0c3aa77 2023-07-30 23:56:15,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/.tmp/cf1/fb344aa3832f4680b9d1d96be0c3aa77 as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/fb344aa3832f4680b9d1d96be0c3aa77 2023-07-30 23:56:15,065 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:15,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb344aa3832f4680b9d1d96be0c3aa77 2023-07-30 23:56:15,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/fb344aa3832f4680b9d1d96be0c3aa77, entries=50, sequenceid=13, filesize=6.4 K 2023-07-30 23:56:15,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 6494f975164328a819a48f1beabe91f9 in 33ms, sequenceid=13, compaction requested=false 2023-07-30 23:56:15,090 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:15,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/recovered.edits/16.seqid, newMaxSeqId=16, maxSeqId=9 2023-07-30 23:56:15,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:15,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:56:15,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:15,113 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46759,1690761342152; all regions closed. 2023-07-30 23:56:15,124 DEBUG [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:56:15,124 INFO [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46759%2C1690761342152.meta:.meta(num 1690761342680) 2023-07-30 23:56:15,134 DEBUG [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:56:15,134 INFO [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46759%2C1690761342152.jenkins-hbase4.apache.org%2C46759%2C1690761342152.regiongroup-0:(num 1690761342531) 2023-07-30 23:56:15,139 DEBUG [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:56:15,139 INFO [RS:0;jenkins-hbase4:46759] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46759%2C1690761342152.jenkins-hbase4.apache.org%2C46759%2C1690761342152.regiongroup-1:(num 1690761347235) 2023-07-30 23:56:15,139 DEBUG [RS:0;jenkins-hbase4:46759] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,139 INFO [RS:0;jenkins-hbase4:46759] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:56:15,140 INFO [RS:0;jenkins-hbase4:46759] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-30 23:56:15,140 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:56:15,140 DEBUG [RS:0;jenkins-hbase4:46759] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2dc21def to 127.0.0.1:52585 2023-07-30 23:56:15,140 DEBUG [RS:0;jenkins-hbase4:46759] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,141 INFO [RS:0;jenkins-hbase4:46759] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46759 2023-07-30 23:56:15,146 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90029, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:15,146 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46759,1690761342152 2023-07-30 23:56:15,146 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:15,146 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:15,146 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:15,146 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90026, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:15,146 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:15,148 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46759,1690761342152] 2023-07-30 23:56:15,148 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46759,1690761342152; numProcessing=1 2023-07-30 23:56:15,150 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46759,1690761342152 already deleted, retry=false 2023-07-30 23:56:15,150 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46759,1690761342152 expired; onlineServers=0 2023-07-30 23:56:15,150 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37557,1690761342123' ***** 2023-07-30 23:56:15,150 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-30 23:56:15,151 DEBUG [M:0;jenkins-hbase4:37557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3df8b142, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:56:15,151 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37557,1690761342123 2023-07-30 23:56:15,151 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37557,1690761342123; all regions closed. 2023-07-30 23:56:15,151 DEBUG [M:0;jenkins-hbase4:37557] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,151 DEBUG [M:0;jenkins-hbase4:37557] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-30 23:56:15,151 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-30 23:56:15,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761342351] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761342351,5,FailOnTimeoutGroup] 2023-07-30 23:56:15,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761342351] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761342351,5,FailOnTimeoutGroup] 2023-07-30 23:56:15,151 DEBUG [M:0;jenkins-hbase4:37557] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-30 23:56:15,154 INFO [M:0;jenkins-hbase4:37557] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-30 23:56:15,154 INFO [M:0;jenkins-hbase4:37557] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-30 23:56:15,154 INFO [M:0;jenkins-hbase4:37557] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-30 23:56:15,154 DEBUG [M:0;jenkins-hbase4:37557] master.HMaster(1512): Stopping service threads 2023-07-30 23:56:15,154 INFO [M:0;jenkins-hbase4:37557] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-30 23:56:15,155 ERROR [M:0;jenkins-hbase4:37557] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-30 23:56:15,155 INFO [M:0;jenkins-hbase4:37557] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-30 23:56:15,155 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-30 23:56:15,155 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-30 23:56:15,155 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:56:15,155 DEBUG [M:0;jenkins-hbase4:37557] zookeeper.ZKUtil(398): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-30 23:56:15,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:56:15,155 WARN [M:0;jenkins-hbase4:37557] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-30 23:56:15,155 INFO [M:0;jenkins-hbase4:37557] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-30 23:56:15,157 INFO [M:0;jenkins-hbase4:37557] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-30 23:56:15,157 DEBUG [M:0;jenkins-hbase4:37557] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:56:15,157 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:15,158 DEBUG [M:0;jenkins-hbase4:37557] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:15,158 DEBUG [M:0;jenkins-hbase4:37557] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:56:15,158 DEBUG [M:0;jenkins-hbase4:37557] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:15,158 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.39 KB heapSize=46.76 KB 2023-07-30 23:56:15,171 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:15,183 INFO [M:0;jenkins-hbase4:37557] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.39 KB at sequenceid=238 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/27efdad650fd4232b489eadf9c78801a 2023-07-30 23:56:15,188 INFO [M:0;jenkins-hbase4:37557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27efdad650fd4232b489eadf9c78801a 2023-07-30 23:56:15,189 DEBUG [M:0;jenkins-hbase4:37557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/27efdad650fd4232b489eadf9c78801a as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/27efdad650fd4232b489eadf9c78801a 2023-07-30 23:56:15,197 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:15,198 INFO [M:0;jenkins-hbase4:37557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27efdad650fd4232b489eadf9c78801a 2023-07-30 23:56:15,199 INFO [M:0;jenkins-hbase4:37557] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/27efdad650fd4232b489eadf9c78801a, entries=20, sequenceid=238, filesize=5.6 K 2023-07-30 23:56:15,203 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegion(2948): Finished flush of dataSize ~36.39 KB/37259, heapSize ~46.74 KB/47864, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 45ms, sequenceid=238, compaction requested=false 2023-07-30 23:56:15,205 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:15,205 DEBUG [M:0;jenkins-hbase4:37557] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:56:15,210 INFO [M:0;jenkins-hbase4:37557] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-30 23:56:15,210 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:56:15,211 INFO [M:0;jenkins-hbase4:37557] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37557 2023-07-30 23:56:15,214 DEBUG [M:0;jenkins-hbase4:37557] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,37557,1690761342123 already deleted, retry=false 2023-07-30 23:56:15,276 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:15,302 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:15,380 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:15,406 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:15,418 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:15,418 INFO [M:0;jenkins-hbase4:37557] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37557,1690761342123; zookeeper connection closed. 2023-07-30 23:56:15,418 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:37557-0x101b8a8e9d9001c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:15,484 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:15,510 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 791 2023-07-30 23:56:15,518 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:15,518 INFO [RS:0;jenkins-hbase4:46759] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46759,1690761342152; zookeeper connection closed. 2023-07-30 23:56:15,518 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:46759-0x101b8a8e9d9001d, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:15,519 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@46341b04] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@46341b04 2023-07-30 23:56:15,520 INFO [Listener at localhost/42375] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-30 23:56:15,523 DEBUG [hconnection-0x30b53367-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:15,525 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46508, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:56:15,589 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 454 2023-07-30 23:56:15,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9341 2023-07-30 23:56:15,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 81 WAL entries eligible for replication 2023-07-30 23:56:15,623 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9341 2023-07-30 23:56:15,623 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/409ccc1344e052df313f0ebb0776f6ff/112=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/113=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/114=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/115=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/116=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/117=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/118=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/119=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/120=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/121=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/122=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/123=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/124=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/125=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/126=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/127=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/128=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/129=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/130=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/131=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/132=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/133=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/134=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/135=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/136=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/137=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/138=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/139=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/140=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/141=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/142=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/143=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/144=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/145=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/146=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/147=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/148=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/149=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/150=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/151=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/152=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/153=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/154=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/155=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/156=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/157=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/158=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/159=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/160=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/161=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/162=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/163=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/164=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/165=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/166=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/167=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/168=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/169=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/170=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/171=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/172=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/173=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/174=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/175=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/176=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/177=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/178=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/179=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/180=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/181=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/182=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/183=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/184=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/185=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/186=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/187=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/188=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/189=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/190=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/191=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/192=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973, lastWalPosition=9341, nbRowKeys=81, nbHFiles=0, heapSize=16848, lastSeqIds={}, endOfFile=false,usedBufferSize=16848] 2023-07-30 23:56:15,626 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:15,696 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 3720 2023-07-30 23:56:15,696 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 31 WAL entries eligible for replication 2023-07-30 23:56:15,696 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 3720 2023-07-30 23:56:15,696 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/212=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/213=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/214=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/215=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/216=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/217=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/218=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/219=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/220=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/221=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/222=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/223=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/224=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/225=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/226=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/227=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/228=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/229=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/230=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/231=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/232=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/233=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/234=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/235=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/236=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/237=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/238=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/239=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/240=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/241=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/242=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/WALs/jenkins-hbase4.apache.org,43785,1690761368937/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497, lastWalPosition=3720, nbRowKeys=31, nbHFiles=0, heapSize=6448, lastSeqIds={}, endOfFile=false,usedBufferSize=6448] 2023-07-30 23:56:15,698 WARN [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:15,726 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:15,736 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-07-30 23:56:15,736 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 19 WAL entries eligible for replication 2023-07-30 23:56:15,736 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-07-30 23:56:15,799 WARN [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:15,812 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 12654 2023-07-30 23:56:15,812 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 84 WAL entries eligible for replication 2023-07-30 23:56:15,812 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 12654 2023-07-30 23:56:15,841 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 11467 2023-07-30 23:56:15,908 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c775526 to 127.0.0.1:52585 2023-07-30 23:56:15,909 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,909 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-30 23:56:15,909 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(257): Found active master hash=328427290, stopped=false 2023-07-30 23:56:15,909 INFO [Listener at localhost/42375] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:15,911 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-30 23:56:15,911 INFO [Listener at localhost/42375] procedure2.ProcedureExecutor(629): Stopping 2023-07-30 23:56:15,911 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-30 23:56:15,911 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:56:15,914 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2eb892d5 to 127.0.0.1:52585 2023-07-30 23:56:15,914 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:56:15,914 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43785,1690761368937' ***** 2023-07-30 23:56:15,914 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:56:15,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-30 23:56:15,914 INFO [RS:0;jenkins-hbase4:43785] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:56:15,915 INFO [RS:0;jenkins-hbase4:43785] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:56:15,915 INFO [RS:0;jenkins-hbase4:43785] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:56:15,915 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(3305): Received CLOSE for ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:15,915 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:56:15,916 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(3305): Received CLOSE for 409ccc1344e052df313f0ebb0776f6ff 2023-07-30 23:56:15,917 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(3305): Received CLOSE for bffda5817d519b05008d136099bf920f 2023-07-30 23:56:15,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ef4a62973dc7531db8626f8122ae55c2, disabling compactions & flushes 2023-07-30 23:56:15,917 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:15,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:15,918 DEBUG [RS:0;jenkins-hbase4:43785] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0b0c92b1 to 127.0.0.1:52585 2023-07-30 23:56:15,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:15,918 DEBUG [RS:0;jenkins-hbase4:43785] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:15,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. after waiting 0 ms 2023-07-30 23:56:15,918 INFO [RS:0;jenkins-hbase4:43785] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:56:15,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:15,918 INFO [RS:0;jenkins-hbase4:43785] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:56:15,918 INFO [RS:0;jenkins-hbase4:43785] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:56:15,918 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-30 23:56:15,918 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-30 23:56:15,918 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1478): Online Regions={ef4a62973dc7531db8626f8122ae55c2=hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2., 1588230740=hbase:meta,,1.1588230740, 409ccc1344e052df313f0ebb0776f6ff=t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff., bffda5817d519b05008d136099bf920f=t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f.} 2023-07-30 23:56:15,919 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1504): Waiting on 1588230740, 409ccc1344e052df313f0ebb0776f6ff, bffda5817d519b05008d136099bf920f, ef4a62973dc7531db8626f8122ae55c2 2023-07-30 23:56:15,919 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:56:15,919 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:56:15,919 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:56:15,919 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:56:15,919 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:56:15,919 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-30 23:56:15,924 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/namespace/ef4a62973dc7531db8626f8122ae55c2/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-30 23:56:15,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:15,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ef4a62973dc7531db8626f8122ae55c2: 2023-07-30 23:56:15,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690761331656.ef4a62973dc7531db8626f8122ae55c2. 2023-07-30 23:56:15,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 409ccc1344e052df313f0ebb0776f6ff, disabling compactions & flushes 2023-07-30 23:56:15,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:15,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:15,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. after waiting 0 ms 2023-07-30 23:56:15,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:15,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 409ccc1344e052df313f0ebb0776f6ff 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-30 23:56:15,928 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:15,928 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21946 2023-07-30 23:56:15,928 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 85 WAL entries eligible for replication 2023-07-30 23:56:15,935 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/info/c204ddb78d3b47b694c0678165b92436 2023-07-30 23:56:15,948 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 12016 2023-07-30 23:56:15,948 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-30 23:56:15,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/cf1/0d05354bad404dff80591eae83054100 2023-07-30 23:56:15,956 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/rep_barrier/76b3ffe95c2f4f988b40fd2c632435e0 2023-07-30 23:56:15,956 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:56:15,964 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/info/c204ddb78d3b47b694c0678165b92436 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info/c204ddb78d3b47b694c0678165b92436 2023-07-30 23:56:15,970 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/info/c204ddb78d3b47b694c0678165b92436, entries=24, sequenceid=31, filesize=7.4 K 2023-07-30 23:56:15,971 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/.tmp/rep_barrier/76b3ffe95c2f4f988b40fd2c632435e0 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier/76b3ffe95c2f4f988b40fd2c632435e0 2023-07-30 23:56:15,977 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/rep_barrier/76b3ffe95c2f4f988b40fd2c632435e0, entries=2, sequenceid=31, filesize=4.9 K 2023-07-30 23:56:15,978 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 59ms, sequenceid=31, compaction requested=false 2023-07-30 23:56:15,985 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-30 23:56:15,986 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-30 23:56:15,986 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:56:15,986 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:56:15,986 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-30 23:56:16,000 WARN [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:16,119 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1504): Waiting on 409ccc1344e052df313f0ebb0776f6ff, bffda5817d519b05008d136099bf920f 2023-07-30 23:56:16,228 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:16,301 WARN [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-30 23:56:16,319 DEBUG [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1504): Waiting on 409ccc1344e052df313f0ebb0776f6ff, bffda5817d519b05008d136099bf920f 2023-07-30 23:56:16,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/norep/da1f66fe6c75486f87d618ca75aaf63e 2023-07-30 23:56:16,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/cf1/0d05354bad404dff80591eae83054100 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1/0d05354bad404dff80591eae83054100 2023-07-30 23:56:16,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/cf1/0d05354bad404dff80591eae83054100, entries=100, sequenceid=213, filesize=8.4 K 2023-07-30 23:56:16,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/.tmp/norep/da1f66fe6c75486f87d618ca75aaf63e as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep/da1f66fe6c75486f87d618ca75aaf63e 2023-07-30 23:56:16,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/norep/da1f66fe6c75486f87d618ca75aaf63e, entries=1, sequenceid=213, filesize=4.8 K 2023-07-30 23:56:16,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for 409ccc1344e052df313f0ebb0776f6ff in 470ms, sequenceid=213, compaction requested=false 2023-07-30 23:56:16,399 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-30 23:56:16,400 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-30 23:56:16,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t1_syncup/409ccc1344e052df313f0ebb0776f6ff/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-30 23:56:16,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:16,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 409ccc1344e052df313f0ebb0776f6ff: 2023-07-30 23:56:16,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1690761335380.409ccc1344e052df313f0ebb0776f6ff. 2023-07-30 23:56:16,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bffda5817d519b05008d136099bf920f, disabling compactions & flushes 2023-07-30 23:56:16,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:16,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:16,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. after waiting 0 ms 2023-07-30 23:56:16,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:16,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing bffda5817d519b05008d136099bf920f 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-30 23:56:16,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/cf1/780ddf079ae6445cb4fd511d9249f58c 2023-07-30 23:56:16,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/norep/2883a097c9c44050be0f2342d8498134 2023-07-30 23:56:16,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/cf1/780ddf079ae6445cb4fd511d9249f58c as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1/780ddf079ae6445cb4fd511d9249f58c 2023-07-30 23:56:16,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/cf1/780ddf079ae6445cb4fd511d9249f58c, entries=200, sequenceid=413, filesize=12.5 K 2023-07-30 23:56:16,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/.tmp/norep/2883a097c9c44050be0f2342d8498134 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep/2883a097c9c44050be0f2342d8498134 2023-07-30 23:56:16,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/norep/2883a097c9c44050be0f2342d8498134, entries=1, sequenceid=413, filesize=4.8 K 2023-07-30 23:56:16,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for bffda5817d519b05008d136099bf920f in 64ms, sequenceid=413, compaction requested=false 2023-07-30 23:56:16,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/data/default/t2_syncup/bffda5817d519b05008d136099bf920f/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-30 23:56:16,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:16,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bffda5817d519b05008d136099bf920f: 2023-07-30 23:56:16,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1690761336017.bffda5817d519b05008d136099bf920f. 2023-07-30 23:56:16,520 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43785,1690761368937; all regions closed. 2023-07-30 23:56:16,527 DEBUG [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:56:16,527 INFO [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43785%2C1690761368937.meta:.meta(num 1690761369587) 2023-07-30 23:56:16,533 DEBUG [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:56:16,533 INFO [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1:(num 1690761373973) 2023-07-30 23:56:16,539 DEBUG [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs 2023-07-30 23:56:16,539 INFO [RS:0;jenkins-hbase4:43785] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0:(num 1690761369497) 2023-07-30 23:56:16,539 DEBUG [RS:0;jenkins-hbase4:43785] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:16,539 INFO [RS:0;jenkins-hbase4:43785] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:56:16,539 INFO [RS:0;jenkins-hbase4:43785] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-30 23:56:16,539 INFO [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-30 23:56:16,540 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:56:16,629 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-30 23:56:16,648 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d9002f, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:16,648 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d9002f, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:16,648 DEBUG [RS:0;jenkins-hbase4:43785] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70bff179 to 127.0.0.1:52585 2023-07-30 23:56:16,649 DEBUG [RS:0;jenkins-hbase4:43785] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:16,751 DEBUG [RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-07-30 23:56:16,850 INFO [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:43785.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1 terminated 2023-07-30 23:56:16,851 INFO [RS:0;jenkins-hbase4:43785] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1 terminated 2023-07-30 23:56:16,851 INFO [RS:0;jenkins-hbase4:43785] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43785 2023-07-30 23:56:16,855 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:16,855 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:56:16,855 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-30 23:56:16,856 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:16,857 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43785,1690761368937; numProcessing=1 2023-07-30 23:56:16,859 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,43785,1690761368937 already deleted, retry=false 2023-07-30 23:56:16,859 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43785,1690761368937 expired; onlineServers=0 2023-07-30 23:56:16,859 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45867,1690761368875' ***** 2023-07-30 23:56:16,859 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-30 23:56:16,860 DEBUG [M:0;jenkins-hbase4:45867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17acf1b0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:56:16,860 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45867,1690761368875 2023-07-30 23:56:16,860 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45867,1690761368875; all regions closed. 2023-07-30 23:56:16,860 DEBUG [M:0;jenkins-hbase4:45867] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:16,860 DEBUG [M:0;jenkins-hbase4:45867] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-30 23:56:16,860 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-30 23:56:16,860 DEBUG [M:0;jenkins-hbase4:45867] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-30 23:56:16,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761369241] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761369241,5,FailOnTimeoutGroup] 2023-07-30 23:56:16,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761369240] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761369240,5,FailOnTimeoutGroup] 2023-07-30 23:56:16,861 INFO [M:0;jenkins-hbase4:45867] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-30 23:56:16,862 INFO [M:0;jenkins-hbase4:45867] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-30 23:56:16,862 INFO [M:0;jenkins-hbase4:45867] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-30 23:56:16,862 DEBUG [M:0;jenkins-hbase4:45867] master.HMaster(1512): Stopping service threads 2023-07-30 23:56:16,862 INFO [M:0;jenkins-hbase4:45867] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-30 23:56:16,862 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-30 23:56:16,863 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-30 23:56:16,863 ERROR [M:0;jenkins-hbase4:45867] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-30 23:56:16,863 INFO [M:0;jenkins-hbase4:45867] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-30 23:56:16,863 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-30 23:56:16,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-30 23:56:16,863 DEBUG [M:0;jenkins-hbase4:45867] zookeeper.ZKUtil(398): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-30 23:56:16,864 WARN [M:0;jenkins-hbase4:45867] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-30 23:56:16,864 INFO [M:0;jenkins-hbase4:45867] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-30 23:56:16,864 INFO [M:0;jenkins-hbase4:45867] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-30 23:56:16,866 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:56:16,866 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:16,866 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:16,866 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:56:16,866 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:16,866 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.95 KB 2023-07-30 23:56:16,877 INFO [M:0;jenkins-hbase4:45867] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a552bb26fa5847269c97dcdc705f1c91 2023-07-30 23:56:16,882 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a552bb26fa5847269c97dcdc705f1c91 as hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a552bb26fa5847269c97dcdc705f1c91 2023-07-30 23:56:16,886 INFO [M:0;jenkins-hbase4:45867] regionserver.HStore(1080): Added hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a552bb26fa5847269c97dcdc705f1c91, entries=10, sequenceid=218, filesize=5.8 K 2023-07-30 23:56:16,886 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30423, heapSize ~36.94 KB/37824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=218, compaction requested=false 2023-07-30 23:56:16,888 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:16,888 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:56:16,894 INFO [M:0;jenkins-hbase4:45867] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-30 23:56:16,894 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:56:16,894 INFO [M:0;jenkins-hbase4:45867] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45867 2023-07-30 23:56:16,896 DEBUG [M:0;jenkins-hbase4:45867] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,45867,1690761368875 already deleted, retry=false 2023-07-30 23:56:16,957 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:16,957 INFO [RS:0;jenkins-hbase4:43785] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43785,1690761368937; zookeeper connection closed. 2023-07-30 23:56:16,957 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:43785-0x101b8a8e9d9002b, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:16,958 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@f8b1a41] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@f8b1a41 2023-07-30 23:56:16,958 INFO [Listener at localhost/42375] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-30 23:56:17,057 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:17,057 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45867,1690761368875; zookeeper connection closed. 2023-07-30 23:56:17,057 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:45867-0x101b8a8e9d9002a, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:17,060 INFO [Listener at localhost/42375] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:56:17,060 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:17,061 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:17,061 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:56:17,061 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:17,061 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:56:17,061 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:56:17,061 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34133 2023-07-30 23:56:17,062 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:17,063 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:17,064 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34133 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:17,072 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:341330x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:17,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34133-0x101b8a8e9d90032 connected 2023-07-30 23:56:17,075 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:56:17,075 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:56:17,075 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:56:17,077 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34133 2023-07-30 23:56:17,078 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34133 2023-07-30 23:56:17,081 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34133 2023-07-30 23:56:17,082 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34133 2023-07-30 23:56:17,085 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34133 2023-07-30 23:56:17,085 INFO [Listener at localhost/42375] master.HMaster(444): hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc, hbase.cluster.distributed=false 2023-07-30 23:56:17,086 DEBUG [pool-662-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-30 23:56:17,100 INFO [Listener at localhost/42375] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-30 23:56:17,101 INFO [Listener at localhost/42375] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39817 2023-07-30 23:56:17,101 INFO [Listener at localhost/42375] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-30 23:56:17,103 DEBUG [Listener at localhost/42375] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-30 23:56:17,103 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:17,104 INFO [Listener at localhost/42375] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:17,105 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39817 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:17,107 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:398170x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:17,109 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:398170x0, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:56:17,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39817-0x101b8a8e9d90033 connected 2023-07-30 23:56:17,110 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:56:17,110 DEBUG [Listener at localhost/42375] zookeeper.ZKUtil(164): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-30 23:56:17,111 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39817 2023-07-30 23:56:17,111 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39817 2023-07-30 23:56:17,111 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39817 2023-07-30 23:56:17,112 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39817 2023-07-30 23:56:17,112 DEBUG [Listener at localhost/42375] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39817 2023-07-30 23:56:17,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,114 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-30 23:56:17,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,117 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:56:17,117 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-30 23:56:17,117 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:56:17,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-30 23:56:17,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,34133,1690761377060 from backup master directory 2023-07-30 23:56:17,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-30 23:56:17,120 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,120 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:56:17,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,120 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-30 23:56:17,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-30 23:56:17,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x76caf104 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:17,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68f4dcc3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:17,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-30 23:56:17,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-30 23:56:17,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:17,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:17,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123 to hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123-dead as it is dead 2023-07-30 23:56:17,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123-dead/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 2023-07-30 23:56:17,176 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123-dead/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 after 1ms 2023-07-30 23:56:17,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123-dead/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 to hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 2023-07-30 23:56:17,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,37557,1690761342123-dead 2023-07-30 23:56:17,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,178 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:17,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34133%2C1690761377060.jenkins-hbase4.apache.org%2C34133%2C1690761377060.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,34133,1690761377060, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/oldWALs, maxLogs=10 2023-07-30 23:56:17,203 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:56:17,207 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:56:17,207 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:56:17,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/WALs/jenkins-hbase4.apache.org,34133,1690761377060/jenkins-hbase4.apache.org%2C34133%2C1690761377060.jenkins-hbase4.apache.org%2C34133%2C1690761377060.regiongroup-0.1690761377180 2023-07-30 23:56:17,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK]] 2023-07-30 23:56:17,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:17,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:17,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:17,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:17,213 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:17,214 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-30 23:56:17,215 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-30 23:56:17,220 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27efdad650fd4232b489eadf9c78801a 2023-07-30 23:56:17,220 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/27efdad650fd4232b489eadf9c78801a 2023-07-30 23:56:17,224 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/906dafa714534cac8f768868161da0a3 2023-07-30 23:56:17,224 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:17,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-30 23:56:17,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 2023-07-30 23:56:17,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 139, firstSequenceIdInLog=120, maxSequenceIdInLog=240, path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 2023-07-30 23:56:17,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C37557%2C1690761342123.jenkins-hbase4.apache.org%2C37557%2C1690761342123.regiongroup-0.1690761342239 2023-07-30 23:56:17,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-30 23:56:17,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/240.seqid, newMaxSeqId=240, maxSeqId=118 2023-07-30 23:56:17,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=241; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61836581, jitterRate=-0.0785631388425827}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:56:17,240 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:56:17,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-30 23:56:17,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-30 23:56:17,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-30 23:56:17,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-30 23:56:17,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-30 23:56:17,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-30 23:56:17,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-30 23:56:17,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-30 23:56:17,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-30 23:56:17,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-30 23:56:17,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-30 23:56:17,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-07-30 23:56:17,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-30 23:56:17,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-30 23:56:17,263 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46759,1690761342152, table=hbase:meta, region=1588230740 2023-07-30 23:56:17,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-30 23:56:17,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46759,1690761342152 already deleted, retry=false 2023-07-30 23:56:17,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46759,1690761342152 on jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46759,1690761342152, splitWal=true, meta=true 2023-07-30 23:56:17,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,46759,1690761342152 (carryingMeta=true) jenkins-hbase4.apache.org,46759,1690761342152/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@e910eae[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-30 23:56:17,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-30 23:56:17,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-30 23:56:17,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-30 23:56:17,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-30 23:56:17,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-30 23:56:17,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-30 23:56:17,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-30 23:56:17,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-30 23:56:17,284 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:56:17,284 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-30 23:56:17,284 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:56:17,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34133,1690761377060, sessionid=0x101b8a8e9d90032, setting cluster-up flag (Was=false) 2023-07-30 23:56:17,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-30 23:56:17,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-30 23:56:17,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:17,295 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/.hbase-snapshot/.tmp 2023-07-30 23:56:17,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:17,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:17,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:17,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-30 23:56:17,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-30 23:56:17,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:56:17,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,333 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(951): ClusterId : 08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:56:17,333 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-30 23:56:17,336 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-30 23:56:17,337 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-30 23:56:17,338 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-30 23:56:17,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690761407349 2023-07-30 23:56:17,350 DEBUG [RS:0;jenkins-hbase4:39817] zookeeper.ReadOnlyZKClient(139): Connect 0x3812808a to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:17,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-30 23:56:17,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-30 23:56:17,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-30 23:56:17,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-30 23:56:17,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-30 23:56:17,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-30 23:56:17,353 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46759,1690761342152; numProcessing=1 2023-07-30 23:56:17,353 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46759,1690761342152, splitWal=true, meta=true 2023-07-30 23:56:17,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,355 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46759,1690761342152, splitWal=true, meta=true, isMeta: true 2023-07-30 23:56:17,357 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152-splitting 2023-07-30 23:56:17,358 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152-splitting dir is empty, no logs to split. 2023-07-30 23:56:17,358 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46759,1690761342152 WAL count=0, meta=true 2023-07-30 23:56:17,361 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152-splitting dir is empty, no logs to split. 2023-07-30 23:56:17,361 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46759,1690761342152 WAL count=0, meta=true 2023-07-30 23:56:17,361 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46759,1690761342152 WAL splitting is done? wals=0, meta=true 2023-07-30 23:56:17,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-30 23:56:17,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-30 23:56:17,367 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-30 23:56:17,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-30 23:56:17,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-30 23:56:17,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-30 23:56:17,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-30 23:56:17,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761377371,5,FailOnTimeoutGroup] 2023-07-30 23:56:17,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761377376,5,FailOnTimeoutGroup] 2023-07-30 23:56:17,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-30 23:56:17,387 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:17,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,387 DEBUG [RS:0;jenkins-hbase4:39817] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36d579f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:17,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,387 DEBUG [RS:0;jenkins-hbase4:39817] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32eaebfa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:56:17,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690761377387, completionTime=-1 2023-07-30 23:56:17,387 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-30 23:56:17,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-30 23:56:17,395 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39817 2023-07-30 23:56:17,395 INFO [RS:0;jenkins-hbase4:39817] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-30 23:56:17,395 INFO [RS:0;jenkins-hbase4:39817] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-30 23:56:17,396 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1022): About to register with Master. 2023-07-30 23:56:17,396 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34133,1690761377060 with isa=jenkins-hbase4.apache.org/172.31.14.131:39817, startcode=1690761377099 2023-07-30 23:56:17,396 DEBUG [RS:0;jenkins-hbase4:39817] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-30 23:56:17,398 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45253, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-30 23:56:17,399 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34133] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,400 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc 2023-07-30 23:56:17,400 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35081 2023-07-30 23:56:17,400 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-30 23:56:17,408 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90026, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:17,408 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90029, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:17,408 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:17,408 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:17,408 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:17,408 DEBUG [RS:0;jenkins-hbase4:39817] zookeeper.ZKUtil(162): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,408 WARN [RS:0;jenkins-hbase4:39817] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-30 23:56:17,408 INFO [RS:0;jenkins-hbase4:39817] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:17,409 INFO [RS:0;jenkins-hbase4:39817] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:17,409 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,410 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39817,1690761377099] 2023-07-30 23:56:17,414 DEBUG [RS:0;jenkins-hbase4:39817] zookeeper.ZKUtil(162): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,415 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:56:17,415 INFO [RS:0;jenkins-hbase4:39817] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-30 23:56:17,417 INFO [RS:0;jenkins-hbase4:39817] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-30 23:56:17,421 INFO [RS:0;jenkins-hbase4:39817] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-30 23:56:17,421 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,421 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-30 23:56:17,423 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,423 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,424 DEBUG [RS:0;jenkins-hbase4:39817] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-30 23:56:17,425 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,425 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,426 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,437 INFO [RS:0;jenkins-hbase4:39817] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-30 23:56:17,437 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39817,1690761377099-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:17,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-30 23:56:17,447 INFO [RS:0;jenkins-hbase4:39817] regionserver.Replication(203): jenkins-hbase4.apache.org,39817,1690761377099 started 2023-07-30 23:56:17,447 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39817,1690761377099, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39817, sessionid=0x101b8a8e9d90033 2023-07-30 23:56:17,447 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-30 23:56:17,447 DEBUG [RS:0;jenkins-hbase4:39817] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39817,1690761377099' 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39817,1690761377099' 2023-07-30 23:56:17,448 DEBUG [RS:0;jenkins-hbase4:39817] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-30 23:56:17,449 DEBUG [RS:0;jenkins-hbase4:39817] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-30 23:56:17,449 DEBUG [RS:0;jenkins-hbase4:39817] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-30 23:56:17,449 INFO [RS:0;jenkins-hbase4:39817] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-30 23:56:17,449 INFO [RS:0;jenkins-hbase4:39817] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-30 23:56:17,537 DEBUG [jenkins-hbase4:34133] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-30 23:56:17,540 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39817,1690761377099, state=OPENING 2023-07-30 23:56:17,542 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-30 23:56:17,542 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-30 23:56:17,542 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39817,1690761377099}] 2023-07-30 23:56:17,550 INFO [RS:0;jenkins-hbase4:39817] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:17,551 INFO [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39817%2C1690761377099.jenkins-hbase4.apache.org%2C39817%2C1690761377099.regiongroup-0, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:56:17,571 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:56:17,575 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:56:17,575 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:56:17,581 INFO [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099/jenkins-hbase4.apache.org%2C39817%2C1690761377099.jenkins-hbase4.apache.org%2C39817%2C1690761377099.regiongroup-0.1690761377552 2023-07-30 23:56:17,584 DEBUG [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK]] 2023-07-30 23:56:17,698 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:17,698 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:56:17,700 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45058, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:56:17,706 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-30 23:56:17,706 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:17,706 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:17,707 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:17,709 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39817%2C1690761377099.meta, suffix=.meta, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:56:17,728 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:56:17,728 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:56:17,728 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK] 2023-07-30 23:56:17,737 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099/jenkins-hbase4.apache.org%2C39817%2C1690761377099.meta.1690761377709.meta 2023-07-30 23:56:17,742 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK], DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:34927,DS-6286f008-7c89-4243-9f75-4f0cf659f3ae,DISK]] 2023-07-30 23:56:17,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:17,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-30 23:56:17,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-30 23:56:17,743 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-30 23:56:17,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-30 23:56:17,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:17,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-30 23:56:17,744 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-30 23:56:17,745 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-30 23:56:17,746 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:56:17,746 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info 2023-07-30 23:56:17,747 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-30 23:56:17,761 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/11fef2fd43a54b52a3a56fd6edead3d8 2023-07-30 23:56:17,769 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/c8dd0467bc824bef8ca79cd38a517f7e 2023-07-30 23:56:17,769 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:17,769 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-30 23:56:17,770 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:56:17,770 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/rep_barrier 2023-07-30 23:56:17,770 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-30 23:56:17,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:17,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-30 23:56:17,772 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:56:17,772 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table 2023-07-30 23:56:17,773 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-30 23:56:17,785 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/table/1f119aa9b99642ff9ae26cb85132b1b4 2023-07-30 23:56:17,785 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:17,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:56:17,789 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740 2023-07-30 23:56:17,792 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-30 23:56:17,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-30 23:56:17,796 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64024309, jitterRate=-0.04596345126628876}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-30 23:56:17,796 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-30 23:56:17,797 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1690761377698 2023-07-30 23:56:17,803 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-30 23:56:17,804 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-30 23:56:17,804 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39817,1690761377099, state=OPEN 2023-07-30 23:56:17,805 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-30 23:56:17,806 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-30 23:56:17,807 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-30 23:56:17,807 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39817,1690761377099 in 264 msec 2023-07-30 23:56:17,810 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-30 23:56:17,812 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 445 msec 2023-07-30 23:56:18,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1555ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1505ms 2023-07-30 23:56:19,481 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:56:20,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3057ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3007ms 2023-07-30 23:56:21,897 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4509ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-30 23:56:21,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-30 23:56:21,900 DEBUG [hconnection-0x6c18c37a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:21,901 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42072, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:56:21,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=51a075540234ea8884d31403732b8710, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46759,1690761342152, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152, openSeqNum=10 2023-07-30 23:56:21,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6494f975164328a819a48f1beabe91f9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46759,1690761342152, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152, openSeqNum=10 2023-07-30 23:56:21,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46759,1690761342152, regionLocation=jenkins-hbase4.apache.org,46759,1690761342152, openSeqNum=12 2023-07-30 23:56:21,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-30 23:56:21,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690761441909 2023-07-30 23:56:21,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690761501909 2023-07-30 23:56:21,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 12 msec 2023-07-30 23:56:21,925 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46759,1690761342152 had 4 regions 2023-07-30 23:56:21,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34133,1690761377060-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:21,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34133,1690761377060-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:21,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34133,1690761377060-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:21,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34133, period=300000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:21,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-30 23:56:21,925 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. is NOT online; state={51a075540234ea8884d31403732b8710 state=OPEN, ts=1690761381908, server=jenkins-hbase4.apache.org,46759,1690761342152}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-30 23:56:21,927 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46759,1690761342152, splitWal=true, meta=true, isMeta: false 2023-07-30 23:56:21,929 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152-splitting dir is empty, no logs to split. 2023-07-30 23:56:21,929 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,46759,1690761342152 WAL count=0, meta=false 2023-07-30 23:56:21,930 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46759,1690761342152/hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., unknown_server=jenkins-hbase4.apache.org,46759,1690761342152/t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9., unknown_server=jenkins-hbase4.apache.org,46759,1690761342152/t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:21,931 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,46759,1690761342152-splitting dir is empty, no logs to split. 2023-07-30 23:56:21,931 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,46759,1690761342152 WAL count=0, meta=false 2023-07-30 23:56:21,931 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46759,1690761342152 WAL splitting is done? wals=0, meta=false 2023-07-30 23:56:21,935 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN}] 2023-07-30 23:56:21,936 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN 2023-07-30 23:56:21,936 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN 2023-07-30 23:56:21,936 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN 2023-07-30 23:56:21,936 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:21,937 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:21,937 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-30 23:56:22,087 DEBUG [jenkins-hbase4:34133] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-30 23:56:22,090 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=51a075540234ea8884d31403732b8710, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:22,091 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:22,091 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=6494f975164328a819a48f1beabe91f9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:22,091 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761382090"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761382090"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761382090"}]},"ts":"1690761382090"} 2023-07-30 23:56:22,091 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761382090"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761382090"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761382090"}]},"ts":"1690761382090"} 2023-07-30 23:56:22,091 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761382090"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690761382090"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690761382090"}]},"ts":"1690761382090"} 2023-07-30 23:56:22,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 51a075540234ea8884d31403732b8710, server=jenkins-hbase4.apache.org,39817,1690761377099}] 2023-07-30 23:56:22,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure ee738a1d08d169b10fce15cc4dc0d043, server=jenkins-hbase4.apache.org,39817,1690761377099}] 2023-07-30 23:56:22,096 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=30, state=RUNNABLE; OpenRegionProcedure 6494f975164328a819a48f1beabe91f9, server=jenkins-hbase4.apache.org,39817,1690761377099}] 2023-07-30 23:56:22,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:22,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-30 23:56:22,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39817%2C1690761377099.jenkins-hbase4.apache.org%2C39817%2C1690761377099.regiongroup-1, suffix=, logDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099, archiveDir=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs, maxLogs=10 2023-07-30 23:56:22,255 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:22,255 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 51a075540234ea8884d31403732b8710, NAME => 'hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:22,255 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,256 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:22,256 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,256 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,257 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,258 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info 2023-07-30 23:56:22,258 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info 2023-07-30 23:56:22,258 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51a075540234ea8884d31403732b8710 columnFamilyName info 2023-07-30 23:56:22,264 DEBUG [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/info/fba57fb05b8f46a0a780d0a7c0f6febf 2023-07-30 23:56:22,264 INFO [StoreOpener-51a075540234ea8884d31403732b8710-1] regionserver.HStore(310): Store=51a075540234ea8884d31403732b8710/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:22,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,266 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,267 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK] 2023-07-30 23:56:22,268 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK] 2023-07-30 23:56:22,268 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK] 2023-07-30 23:56:22,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/WALs/jenkins-hbase4.apache.org,39817,1690761377099/jenkins-hbase4.apache.org%2C39817%2C1690761377099.jenkins-hbase4.apache.org%2C39817%2C1690761377099.regiongroup-1.1690761382253 2023-07-30 23:56:22,271 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 51a075540234ea8884d31403732b8710 2023-07-30 23:56:22,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44771,DS-8821620a-b9fd-4632-848b-c4ba06b4a6c0,DISK], DatanodeInfoWithStorage[127.0.0.1:40315,DS-f652aaa1-b470-4fc3-9d06-0a0a18420452,DISK], DatanodeInfoWithStorage[127.0.0.1:35861,DS-6fc51d82-b663-4106-bddc-c04d3ce1a2d9,DISK]] 2023-07-30 23:56:22,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6494f975164328a819a48f1beabe91f9, NAME => 't1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:22,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:22,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,272 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 51a075540234ea8884d31403732b8710; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65319121, jitterRate=-0.026669248938560486}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-30 23:56:22,273 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:56:22,273 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., pid=33, masterSystemTime=1690761382247 2023-07-30 23:56:22,273 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,275 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1 2023-07-30 23:56:22,275 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1 2023-07-30 23:56:22,276 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6494f975164328a819a48f1beabe91f9 columnFamilyName cf1 2023-07-30 23:56:22,276 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:22,276 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:22,276 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=51a075540234ea8884d31403732b8710, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:22,276 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690761382276"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761382276"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761382276"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761382276"}]},"ts":"1690761382276"} 2023-07-30 23:56:22,279 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-07-30 23:56:22,279 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 51a075540234ea8884d31403732b8710, server=jenkins-hbase4.apache.org,39817,1690761377099 in 184 msec 2023-07-30 23:56:22,280 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=51a075540234ea8884d31403732b8710, ASSIGN in 346 msec 2023-07-30 23:56:22,282 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/00173fc536fb4d1b8a36c88d95b68e51 2023-07-30 23:56:22,287 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb344aa3832f4680b9d1d96be0c3aa77 2023-07-30 23:56:22,287 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/fb344aa3832f4680b9d1d96be0c3aa77 2023-07-30 23:56:22,287 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(310): Store=6494f975164328a819a48f1beabe91f9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:22,287 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,288 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/norep 2023-07-30 23:56:22,288 DEBUG [StoreOpener-6494f975164328a819a48f1beabe91f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/norep 2023-07-30 23:56:22,288 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6494f975164328a819a48f1beabe91f9 columnFamilyName norep 2023-07-30 23:56:22,289 INFO [StoreOpener-6494f975164328a819a48f1beabe91f9-1] regionserver.HStore(310): Store=6494f975164328a819a48f1beabe91f9/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:22,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:56:22,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:22,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6494f975164328a819a48f1beabe91f9; next sequenceid=17; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74509733, jitterRate=0.11028154194355011}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:56:22,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:56:22,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9., pid=35, masterSystemTime=1690761382247 2023-07-30 23:56:22,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:22,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:22,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:22,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee738a1d08d169b10fce15cc4dc0d043, NAME => 't2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.', STARTKEY => '', ENDKEY => ''} 2023-07-30 23:56:22,296 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=6494f975164328a819a48f1beabe91f9, regionState=OPEN, openSeqNum=17, regionLocation=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:22,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,296 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761382296"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761382296"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761382296"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761382296"}]},"ts":"1690761382296"} 2023-07-30 23:56:22,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-30 23:56:22,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,297 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,298 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1 2023-07-30 23:56:22,299 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1 2023-07-30 23:56:22,299 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=30 2023-07-30 23:56:22,299 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=30, state=SUCCESS; OpenRegionProcedure 6494f975164328a819a48f1beabe91f9, server=jenkins-hbase4.apache.org,39817,1690761377099 in 203 msec 2023-07-30 23:56:22,299 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee738a1d08d169b10fce15cc4dc0d043 columnFamilyName cf1 2023-07-30 23:56:22,300 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=6494f975164328a819a48f1beabe91f9, ASSIGN in 366 msec 2023-07-30 23:56:22,305 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/39f56f6009014b15a197bd88c2d5fadc 2023-07-30 23:56:22,309 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd7a2b174c4d4b03b4f3bfc58987fffd 2023-07-30 23:56:22,310 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(539): loaded hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/fd7a2b174c4d4b03b4f3bfc58987fffd 2023-07-30 23:56:22,310 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(310): Store=ee738a1d08d169b10fce15cc4dc0d043/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:22,310 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,311 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/norep 2023-07-30 23:56:22,311 DEBUG [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/norep 2023-07-30 23:56:22,311 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee738a1d08d169b10fce15cc4dc0d043 columnFamilyName norep 2023-07-30 23:56:22,311 INFO [StoreOpener-ee738a1d08d169b10fce15cc4dc0d043-1] regionserver.HStore(310): Store=ee738a1d08d169b10fce15cc4dc0d043/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-30 23:56:22,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-30 23:56:22,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:22,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee738a1d08d169b10fce15cc4dc0d043; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66013049, jitterRate=-0.01632891595363617}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-30 23:56:22,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:56:22,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043., pid=34, masterSystemTime=1690761382247 2023-07-30 23:56:22,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:22,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:22,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=ee738a1d08d169b10fce15cc4dc0d043, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:22,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1690761382319"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690761382319"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690761382319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690761382319"}]},"ts":"1690761382319"} 2023-07-30 23:56:22,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-30 23:56:22,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure ee738a1d08d169b10fce15cc4dc0d043, server=jenkins-hbase4.apache.org,39817,1690761377099 in 226 msec 2023-07-30 23:56:22,324 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=27 2023-07-30 23:56:22,324 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46759,1690761342152 after splitting done 2023-07-30 23:56:22,324 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=ee738a1d08d169b10fce15cc4dc0d043, ASSIGN in 389 msec 2023-07-30 23:56:22,324 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46759,1690761342152 from processing; numProcessing=0 2023-07-30 23:56:22,325 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46759,1690761342152, splitWal=true, meta=true in 5.0580 sec 2023-07-30 23:56:22,447 INFO [jenkins-hbase4:39817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:56:22,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-30 23:56:22,938 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-30 23:56:22,941 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-30 23:56:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.821sec 2023-07-30 23:56:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-30 23:56:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-30 23:56:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-30 23:56:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34133,1690761377060-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-30 23:56:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34133,1690761377060-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-30 23:56:22,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-30 23:56:23,037 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x27142824 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:23,041 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@150c81df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:23,042 DEBUG [hconnection-0x5f0c068d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:23,044 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42082, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:56:23,046 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-30 23:56:23,046 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27142824 to 127.0.0.1:52585 2023-07-30 23:56:23,046 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:23,047 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:34133 after: jenkins-hbase4.apache.org:34133 2023-07-30 23:56:23,048 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(139): Connect 0x6148acb1 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:23,052 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ecfe67b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:23,416 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-30 23:56:23,416 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-30 23:56:23,417 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-30 23:56:23,417 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-30 23:56:23,553 DEBUG [hconnection-0xc225051-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:23,555 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42098, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-30 23:56:23,575 INFO [Listener at localhost/42375] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1690761383574 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:23,577 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication16907613835740x0, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:23,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1690761383574-0x101b8a8e9d90038 connected Start Replication Server start 2023-07-30 23:56:23,579 INFO [Listener at localhost/42375] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-30 23:56:23,579 INFO [Listener at localhost/42375] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-30 23:56:23,591 INFO [Listener at localhost/42375] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-30 23:56:23,593 DEBUG [Listener at localhost/42375] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-30 23:56:23,602 INFO [Listener at localhost/42375] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:56:23,604 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,43785,1690761368937] other RSs: [] 2023-07-30 23:56:23,622 DEBUG [Listener at localhost/42375.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x5b02398f to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:23,626 DEBUG [Listener at localhost/42375.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@22e0eef6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:23,626 INFO [Listener at localhost/42375.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:23,633 DEBUG [Listener at localhost/42375.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:23,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d9003a connected 2023-07-30 23:56:23,635 INFO [Listener at localhost/42375.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:56:27,125 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,43785,1690761368937/1's WALs to 1690761383579.syncuptool.replication.org,1234,1 2023-07-30 23:56:27,127 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497 with data PBUF\x08\xC6\x03 2023-07-30 23:56:27,128 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973 with data PBUF\x08\x97\x06 2023-07-30 23:56:27,129 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,43785,1690761368937/1's WALs to 1690761383579.syncuptool.replication.org,1234,1 2023-07-30 23:56:27,447 INFO [jenkins-hbase4:39817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:56:27,758 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:56:29,131 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:29,131 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:29,142 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:29,142 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,43785,1690761368937, ReplicationSource: 1, currentBandwidth=0 2023-07-30 23:56:29,142 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:29,162 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] zookeeper.ReadOnlyZKClient(139): Connect 0x3ef92852 to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:29,166 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3c4062ce, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:29,167 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52585 2023-07-30 23:56:29,168 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-30 23:56:29,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101b8a8e9d9003c connected 2023-07-30 23:56:29,170 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,43785,1690761368937 (queues=2) is replicating from cluster=c047cc16-085a-45ee-a021-aa0b6e22ca94 to cluster=08550826-6266-4a96-96ef-3a20bffd1026 2023-07-30 23:56:29,170 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0 2023-07-30 23:56:29,172 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,43785,1690761368937, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:56:29,173 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1 2023-07-30 23:56:29,173 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0 2023-07-30 23:56:29,174 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,43785,1690761368937, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-30 23:56:29,175 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1 2023-07-30 23:56:29,184 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973 2023-07-30 23:56:29,184 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceWALReader(162): Read 100 WAL entries eligible for replication 2023-07-30 23:56:29,184 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-30 23:56:29,185 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43785,1690761368937 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/409ccc1344e052df313f0ebb0776f6ff/112=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/113=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/114=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/115=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/116=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/117=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/118=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/119=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/120=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/121=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/122=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/123=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/124=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/125=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/126=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/127=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/128=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/129=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/130=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/131=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/132=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/133=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/134=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/135=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/136=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/137=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/138=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/139=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/140=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/141=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/142=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/143=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/144=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/145=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/146=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/147=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/148=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/149=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/150=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/151=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/152=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/153=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/154=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/155=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/156=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/157=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/158=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/159=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/160=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/161=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/162=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/163=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/164=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/165=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/166=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/167=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/168=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/169=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/170=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/171=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/172=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/173=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/174=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/175=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/176=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/177=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/178=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/179=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/180=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/181=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/182=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/183=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/184=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/185=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/186=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/187=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/188=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/189=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/190=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/191=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/192=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/193=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/194=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/195=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/196=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/197=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/198=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/199=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/200=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/201=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/202=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/203=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/204=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/205=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/206=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/207=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/208=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/209=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/210=[#edits: 1 = ],208}, {t1_syncup/409ccc1344e052df313f0ebb0776f6ff/211=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973, lastWalPosition=12631, nbRowKeys=100, nbHFiles=0, heapSize=20800, lastSeqIds={}, endOfFile=true,usedBufferSize=20800] 2023-07-30 23:56:29,186 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497 2023-07-30 23:56:29,187 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceWALReader(162): Read 200 WAL entries eligible for replication 2023-07-30 23:56:29,187 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-30 23:56:29,188 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43785,1690761368937 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/bffda5817d519b05008d136099bf920f/212=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/213=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/214=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/215=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/216=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/217=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/218=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/219=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/220=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/221=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/222=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/223=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/224=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/225=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/226=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/227=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/228=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/229=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/230=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/231=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/232=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/233=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/234=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/235=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/236=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/237=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/238=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/239=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/240=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/241=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/242=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/243=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/244=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/245=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/246=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/247=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/248=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/249=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/250=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/251=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/252=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/253=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/254=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/255=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/256=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/257=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/258=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/259=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/260=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/261=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/262=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/263=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/264=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/265=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/266=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/267=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/268=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/269=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/270=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/271=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/272=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/273=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/274=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/275=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/276=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/277=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/278=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/279=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/280=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/281=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/282=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/283=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/284=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/285=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/286=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/287=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/288=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/289=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/290=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/291=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/292=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/293=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/294=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/295=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/296=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/297=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/298=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/299=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/300=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/301=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/302=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/303=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/304=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/305=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/306=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/307=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/308=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/309=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/310=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/311=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/312=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/313=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/314=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/315=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/316=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/317=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/318=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/319=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/320=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/321=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/322=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/323=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/324=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/325=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/326=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/327=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/328=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/329=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/330=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/331=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/332=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/333=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/334=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/335=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/336=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/337=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/338=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/339=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/340=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/341=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/342=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/343=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/344=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/345=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/346=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/347=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/348=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/349=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/350=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/351=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/352=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/353=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/354=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/355=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/356=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/357=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/358=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/359=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/360=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/361=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/362=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/363=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/364=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/365=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/366=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/367=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/368=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/369=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/370=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/371=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/372=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/373=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/374=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/375=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/376=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/377=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/378=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/379=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/380=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/381=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/382=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/383=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/384=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/385=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/386=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/387=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/388=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/389=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/390=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/391=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/392=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/393=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/394=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/395=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/396=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/397=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/398=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/399=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/400=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/401=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/402=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/403=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/404=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/405=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/406=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/407=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/408=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/409=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/410=[#edits: 1 = ],208}, {t2_syncup/bffda5817d519b05008d136099bf920f/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45087/user/jenkins/test-data/4227e976-3489-2722-6db2-008abdd6d2e5/oldWALs/jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497, lastWalPosition=22804, nbRowKeys=200, nbHFiles=0, heapSize=41600, lastSeqIds={}, endOfFile=true,usedBufferSize=41600] 2023-07-30 23:56:29,191 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-30 23:56:29,194 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42108, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-30 23:56:29,196 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39817] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:56:29,196 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39817] zookeeper.ReadOnlyZKClient(139): Connect 0x045267bc to 127.0.0.1:52585 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-30 23:56:29,197 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=39817] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-30 23:56:29,200 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39817] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@a959946, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-30 23:56:29,203 DEBUG [hconnection-0xeaa0ccc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-30 23:56:29,204 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42120, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-30 23:56:29,215 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39817] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:56:29,216 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:29,216 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=39817] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-30 23:56:29,217 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43785,1690761368937] 2023-07-30 23:56:29,217 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1.1690761373973] 2023-07-30 23:56:29,221 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0.1690761369497] 2023-07-30 23:56:29,222 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43785,1690761368937 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:56:29,222 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:29,223 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43785,1690761368937 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-30 23:56:29,223 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:29,322 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:29,325 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761383574-0x101b8a8e9d90038, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1690761383579.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:29,325 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-1,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,43785,1690761368937 with the following stats: Total replicated edits: 300, current progress: 2023-07-30 23:56:29,426 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,43785,1690761368937 2023-07-30 23:56:29,426 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] zookeeper.ZKUtil(243): syncupReplication1690761383574-0x101b8a8e9d90038, quorum=127.0.0.1:52585, baseZNode=/1 Unable to list children of znode /1/replication/rs/1690761383579.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43785,1690761368937 because node does not exist (not an error) 2023-07-30 23:56:29,426 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937.replicationSource.shipperjenkins-hbase4.apache.org%2C43785%2C1690761368937.jenkins-hbase4.apache.org%2C43785%2C1690761368937.regiongroup-0,1-jenkins-hbase4.apache.org,43785,1690761368937] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,43785,1690761368937 with the following stats: Total replicated edits: 300, current progress: 2023-07-30 23:56:32,447 INFO [jenkins-hbase4:39817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-30 23:56:32,447 INFO [jenkins-hbase4:39817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-30 23:56:33,604 INFO [Listener at localhost/42375] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-30 23:56:33,707 DEBUG [Listener at localhost/42375.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d9003a, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:33,707 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b02398f to 127.0.0.1:52585 2023-07-30 23:56:33,708 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:33,911 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761383574-0x101b8a8e9d90038, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:33,911 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): syncupReplication1690761383574-0x101b8a8e9d90038, quorum=127.0.0.1:52585, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:33,920 INFO [Listener at localhost/42375] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-30 23:56:33,920 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-30 23:56:33,920 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6148acb1 to 127.0.0.1:52585 2023-07-30 23:56:33,921 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:33,921 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-30 23:56:33,921 DEBUG [Listener at localhost/42375] util.JVMClusterUtil(257): Found active master hash=499730845, stopped=false 2023-07-30 23:56:33,921 INFO [Listener at localhost/42375] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:33,923 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:56:33,923 INFO [Listener at localhost/42375] procedure2.ProcedureExecutor(629): Stopping 2023-07-30 23:56:33,923 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-30 23:56:33,924 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:56:33,924 DEBUG [Listener at localhost/42375] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x76caf104 to 127.0.0.1:52585 2023-07-30 23:56:33,926 DEBUG [Listener at localhost/42375] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:33,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:56:33,926 INFO [Listener at localhost/42375] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39817,1690761377099' ***** 2023-07-30 23:56:33,926 INFO [Listener at localhost/42375] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-30 23:56:33,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-30 23:56:33,926 INFO [RS:0;jenkins-hbase4:39817] regionserver.HeapMemoryManager(220): Stopping 2023-07-30 23:56:33,927 INFO [RS:0;jenkins-hbase4:39817] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-30 23:56:33,927 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-30 23:56:33,927 INFO [RS:0;jenkins-hbase4:39817] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-30 23:56:33,928 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(3305): Received CLOSE for 51a075540234ea8884d31403732b8710 2023-07-30 23:56:33,928 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(3305): Received CLOSE for ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:33,928 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(3305): Received CLOSE for 6494f975164328a819a48f1beabe91f9 2023-07-30 23:56:33,928 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:33,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 51a075540234ea8884d31403732b8710, disabling compactions & flushes 2023-07-30 23:56:33,929 DEBUG [RS:0;jenkins-hbase4:39817] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3812808a to 127.0.0.1:52585 2023-07-30 23:56:33,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:33,929 DEBUG [RS:0;jenkins-hbase4:39817] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:33,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:33,929 INFO [RS:0;jenkins-hbase4:39817] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-30 23:56:33,929 INFO [RS:0;jenkins-hbase4:39817] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-30 23:56:33,929 INFO [RS:0;jenkins-hbase4:39817] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-30 23:56:33,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. after waiting 0 ms 2023-07-30 23:56:33,929 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-30 23:56:33,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:33,929 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-30 23:56:33,929 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1478): Online Regions={51a075540234ea8884d31403732b8710=hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710., ee738a1d08d169b10fce15cc4dc0d043=t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043., 1588230740=hbase:meta,,1.1588230740, 6494f975164328a819a48f1beabe91f9=t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9.} 2023-07-30 23:56:33,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-30 23:56:33,930 DEBUG [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1504): Waiting on 1588230740, 51a075540234ea8884d31403732b8710, 6494f975164328a819a48f1beabe91f9, ee738a1d08d169b10fce15cc4dc0d043 2023-07-30 23:56:33,930 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-30 23:56:33,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-30 23:56:33,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-30 23:56:33,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-30 23:56:33,932 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-30 23:56:33,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/namespace/51a075540234ea8884d31403732b8710/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-30 23:56:33,939 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:56:33,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:33,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 51a075540234ea8884d31403732b8710: 2023-07-30 23:56:33,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690761334623.51a075540234ea8884d31403732b8710. 2023-07-30 23:56:33,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee738a1d08d169b10fce15cc4dc0d043, disabling compactions & flushes 2023-07-30 23:56:33,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:33,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:33,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. after waiting 0 ms 2023-07-30 23:56:33,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:33,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ee738a1d08d169b10fce15cc4dc0d043 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-30 23:56:33,955 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/info/b53269f0947b4fdc8ba508d7dc14665d 2023-07-30 23:56:33,972 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/.tmp/info/b53269f0947b4fdc8ba508d7dc14665d as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/b53269f0947b4fdc8ba508d7dc14665d 2023-07-30 23:56:33,975 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/.tmp/cf1/728bb7e09d184ec794f4515fd8567a1a 2023-07-30 23:56:33,982 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/info/b53269f0947b4fdc8ba508d7dc14665d, entries=24, sequenceid=43, filesize=7.4 K 2023-07-30 23:56:33,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/.tmp/cf1/728bb7e09d184ec794f4515fd8567a1a as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/728bb7e09d184ec794f4515fd8567a1a 2023-07-30 23:56:33,983 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 53ms, sequenceid=43, compaction requested=true 2023-07-30 23:56:33,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/cf1/728bb7e09d184ec794f4515fd8567a1a, entries=200, sequenceid=22, filesize=12.2 K 2023-07-30 23:56:34,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for ee738a1d08d169b10fce15cc4dc0d043 in 61ms, sequenceid=22, compaction requested=true 2023-07-30 23:56:34,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-30 23:56:34,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-30 23:56:34,012 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-30 23:56:34,012 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-30 23:56:34,012 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-30 23:56:34,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t2_syncup/ee738a1d08d169b10fce15cc4dc0d043/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-07-30 23:56:34,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee738a1d08d169b10fce15cc4dc0d043: 2023-07-30 23:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1690761337254.ee738a1d08d169b10fce15cc4dc0d043. 2023-07-30 23:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6494f975164328a819a48f1beabe91f9, disabling compactions & flushes 2023-07-30 23:56:34,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. after waiting 0 ms 2023-07-30 23:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:34,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6494f975164328a819a48f1beabe91f9 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-30 23:56:34,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/.tmp/cf1/e0e14dfe0cd14164a28043e1eb6c3350 2023-07-30 23:56:34,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/.tmp/cf1/e0e14dfe0cd14164a28043e1eb6c3350 as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/e0e14dfe0cd14164a28043e1eb6c3350 2023-07-30 23:56:34,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/cf1/e0e14dfe0cd14164a28043e1eb6c3350, entries=100, sequenceid=20, filesize=8.3 K 2023-07-30 23:56:34,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 6494f975164328a819a48f1beabe91f9 in 23ms, sequenceid=20, compaction requested=true 2023-07-30 23:56:34,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/data/default/t1_syncup/6494f975164328a819a48f1beabe91f9/recovered.edits/23.seqid, newMaxSeqId=23, maxSeqId=16 2023-07-30 23:56:34,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:34,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6494f975164328a819a48f1beabe91f9: 2023-07-30 23:56:34,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1690761336637.6494f975164328a819a48f1beabe91f9. 2023-07-30 23:56:34,130 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39817,1690761377099; all regions closed. 2023-07-30 23:56:34,137 DEBUG [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:56:34,137 INFO [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39817%2C1690761377099.meta:.meta(num 1690761377709) 2023-07-30 23:56:34,142 DEBUG [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:56:34,142 INFO [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39817%2C1690761377099.jenkins-hbase4.apache.org%2C39817%2C1690761377099.regiongroup-0:(num 1690761377552) 2023-07-30 23:56:34,150 DEBUG [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/oldWALs 2023-07-30 23:56:34,150 INFO [RS:0;jenkins-hbase4:39817] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39817%2C1690761377099.jenkins-hbase4.apache.org%2C39817%2C1690761377099.regiongroup-1:(num 1690761382253) 2023-07-30 23:56:34,150 DEBUG [RS:0;jenkins-hbase4:39817] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:34,150 INFO [RS:0;jenkins-hbase4:39817] regionserver.LeaseManager(133): Closed leases 2023-07-30 23:56:34,150 INFO [RS:0;jenkins-hbase4:39817] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-30 23:56:34,151 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:56:34,151 DEBUG [RS:0;jenkins-hbase4:39817] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x045267bc to 127.0.0.1:52585 2023-07-30 23:56:34,151 DEBUG [RS:0;jenkins-hbase4:39817] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:34,152 INFO [RS:0;jenkins-hbase4:39817] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39817 2023-07-30 23:56:34,155 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90029, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:34,155 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:34,155 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:34,155 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,39817,1690761377099 2023-07-30 23:56:34,155 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d9003c, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:34,155 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101b8a8e9d90026, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:34,156 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:34,156 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-30 23:56:34,156 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-30 23:56:34,158 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39817,1690761377099] 2023-07-30 23:56:34,158 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39817,1690761377099; numProcessing=1 2023-07-30 23:56:34,159 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,39817,1690761377099 already deleted, retry=false 2023-07-30 23:56:34,160 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39817,1690761377099 expired; onlineServers=0 2023-07-30 23:56:34,160 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34133,1690761377060' ***** 2023-07-30 23:56:34,160 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-30 23:56:34,160 DEBUG [M:0;jenkins-hbase4:34133] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19f6defc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-30 23:56:34,161 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34133,1690761377060 2023-07-30 23:56:34,161 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34133,1690761377060; all regions closed. 2023-07-30 23:56:34,161 DEBUG [M:0;jenkins-hbase4:34133] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-30 23:56:34,161 DEBUG [M:0;jenkins-hbase4:34133] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-30 23:56:34,161 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-30 23:56:34,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761377371] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690761377371,5,FailOnTimeoutGroup] 2023-07-30 23:56:34,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761377376] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690761377376,5,FailOnTimeoutGroup] 2023-07-30 23:56:34,161 DEBUG [M:0;jenkins-hbase4:34133] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-30 23:56:34,163 INFO [M:0;jenkins-hbase4:34133] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-30 23:56:34,163 INFO [M:0;jenkins-hbase4:34133] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-30 23:56:34,163 INFO [M:0;jenkins-hbase4:34133] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-30 23:56:34,163 DEBUG [M:0;jenkins-hbase4:34133] master.HMaster(1512): Stopping service threads 2023-07-30 23:56:34,163 INFO [M:0;jenkins-hbase4:34133] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-30 23:56:34,164 ERROR [M:0;jenkins-hbase4:34133] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-30 23:56:34,164 INFO [M:0;jenkins-hbase4:34133] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-30 23:56:34,164 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-30 23:56:34,164 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-30 23:56:34,164 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-30 23:56:34,164 DEBUG [M:0;jenkins-hbase4:34133] zookeeper.ZKUtil(398): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-30 23:56:34,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-30 23:56:34,164 WARN [M:0;jenkins-hbase4:34133] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-30 23:56:34,164 INFO [M:0;jenkins-hbase4:34133] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-30 23:56:34,165 INFO [M:0;jenkins-hbase4:34133] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-30 23:56:34,165 DEBUG [M:0;jenkins-hbase4:34133] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-30 23:56:34,165 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:34,166 DEBUG [M:0;jenkins-hbase4:34133] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:34,166 DEBUG [M:0;jenkins-hbase4:34133] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-30 23:56:34,166 DEBUG [M:0;jenkins-hbase4:34133] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:34,166 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-30 23:56:34,181 INFO [M:0;jenkins-hbase4:34133] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/dd2019eee9de44c3bd28140c9faa826f 2023-07-30 23:56:34,187 DEBUG [M:0;jenkins-hbase4:34133] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/dd2019eee9de44c3bd28140c9faa826f as hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/dd2019eee9de44c3bd28140c9faa826f 2023-07-30 23:56:34,192 INFO [M:0;jenkins-hbase4:34133] regionserver.HStore(1080): Added hdfs://localhost:35081/user/jenkins/test-data/ee14dd54-c4d2-55ac-8fa9-f94a4f40e6dc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/dd2019eee9de44c3bd28140c9faa826f, entries=9, sequenceid=309, filesize=5.5 K 2023-07-30 23:56:34,193 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=309, compaction requested=true 2023-07-30 23:56:34,194 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-30 23:56:34,194 DEBUG [M:0;jenkins-hbase4:34133] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-30 23:56:34,199 INFO [M:0;jenkins-hbase4:34133] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-30 23:56:34,200 INFO [M:0;jenkins-hbase4:34133] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34133 2023-07-30 23:56:34,200 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-30 23:56:34,203 DEBUG [M:0;jenkins-hbase4:34133] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,34133,1690761377060 already deleted, retry=false 2023-07-30 23:56:34,323 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:34,324 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): master:34133-0x101b8a8e9d90032, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:34,323 INFO [M:0;jenkins-hbase4:34133] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34133,1690761377060; zookeeper connection closed. 2023-07-30 23:56:34,424 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:34,424 INFO [RS:0;jenkins-hbase4:39817] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39817,1690761377099; zookeeper connection closed. 2023-07-30 23:56:34,424 DEBUG [Listener at localhost/42375-EventThread] zookeeper.ZKWatcher(600): regionserver:39817-0x101b8a8e9d90033, quorum=127.0.0.1:52585, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-30 23:56:34,424 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6e7a3917] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6e7a3917 2023-07-30 23:56:34,424 INFO [Listener at localhost/42375] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-30 23:56:34,425 WARN [Listener at localhost/42375] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-30 23:56:34,430 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:34,535 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-30 23:56:34,535 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-238774971-172.31.14.131-1690761332596 (Datanode Uuid 19c097c5-9cea-4fb8-ba77-caf607a10197) service to localhost/127.0.0.1:35081 2023-07-30 23:56:34,537 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data7/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,537 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data8/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,539 WARN [Listener at localhost/42375] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-30 23:56:34,542 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:34,647 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-30 23:56:34,648 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-238774971-172.31.14.131-1690761332596 (Datanode Uuid a04145b2-4eb9-4c28-8495-692e417b0af2) service to localhost/127.0.0.1:35081 2023-07-30 23:56:34,648 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data5/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,648 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data6/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,650 WARN [Listener at localhost/42375] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-30 23:56:34,653 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:34,757 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-30 23:56:34,757 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-238774971-172.31.14.131-1690761332596 (Datanode Uuid 355d7b89-5ff3-4b64-a399-067a26fed969) service to localhost/127.0.0.1:35081 2023-07-30 23:56:34,757 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data3/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,758 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data4/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,759 WARN [Listener at localhost/42375] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-30 23:56:34,762 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:34,825 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-30 23:56:34,876 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-30 23:56:34,876 WARN [BP-238774971-172.31.14.131-1690761332596 heartbeating to localhost/127.0.0.1:35081] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-238774971-172.31.14.131-1690761332596 (Datanode Uuid b6310d0f-aa61-4bd0-bc5a-06e0b38ddc49) service to localhost/127.0.0.1:35081 2023-07-30 23:56:34,877 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data1/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,877 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d2796dca-12e7-f697-f50d-5af0ac713192/cluster_e57eda41-e1e9-28af-1342-e83ef0792ae5/dfs/data/data2/current/BP-238774971-172.31.14.131-1690761332596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:34,907 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:35,081 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-30 23:56:35,081 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-30 23:56:35,081 WARN [Listener at localhost/42375] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-30 23:56:35,083 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:35,185 WARN [BP-1067226918-172.31.14.131-1690761325473 heartbeating to localhost/127.0.0.1:45087] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-30 23:56:35,185 WARN [BP-1067226918-172.31.14.131-1690761325473 heartbeating to localhost/127.0.0.1:45087] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1067226918-172.31.14.131-1690761325473 (Datanode Uuid 2759b877-2b67-4694-a815-b33dbea45ff5) service to localhost/127.0.0.1:45087 2023-07-30 23:56:35,186 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/dfs/data/data3/current/BP-1067226918-172.31.14.131-1690761325473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:35,186 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/dfs/data/data4/current/BP-1067226918-172.31.14.131-1690761325473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:35,187 WARN [Listener at localhost/42375] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-30 23:56:35,189 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:35,292 WARN [BP-1067226918-172.31.14.131-1690761325473 heartbeating to localhost/127.0.0.1:45087] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-30 23:56:35,292 WARN [BP-1067226918-172.31.14.131-1690761325473 heartbeating to localhost/127.0.0.1:45087] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1067226918-172.31.14.131-1690761325473 (Datanode Uuid e5120aa3-5268-4e18-b3d5-adf3dedbc59f) service to localhost/127.0.0.1:45087 2023-07-30 23:56:35,292 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/dfs/data/data1/current/BP-1067226918-172.31.14.131-1690761325473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:35,293 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a9d64880-8669-c5fa-70c3-5b19427981b4/cluster_0b8392e1-b099-2784-264f-5a4325a76478/dfs/data/data2/current/BP-1067226918-172.31.14.131-1690761325473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-30 23:56:35,302 INFO [Listener at localhost/42375] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-30 23:56:35,417 INFO [Listener at localhost/42375] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-30 23:56:35,442 INFO [Listener at localhost/42375] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-30 23:56:35,465 INFO [Listener at localhost/42375] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=184 (was 10) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (1612332239) connection to localhost/127.0.0.1:35081 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/42375 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1612332239) connection to localhost/127.0.0.1:45087 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x25ef9405 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1009027108.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@374d6215 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x25ef9405-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x3ef92852 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1009027108.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x2db404e1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x3ef92852-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x1838b490-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1612332239) connection to localhost/127.0.0.1:45087 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x3ef92852-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x2db404e1-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: IPC Client (1612332239) connection to localhost/127.0.0.1:35081 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1612332239) connection to localhost/127.0.0.1:35081 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36237,1690761329770-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:45087 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:45087 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x2db404e1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1009027108.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x1838b490 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1009027108.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:45087 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:35081 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43785,1690761368937-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x1838b490-SendThread(127.0.0.1:52585) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:45087 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42313,1690761329724-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52585@0x25ef9405-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) - Thread LEAK? -, OpenFileDescriptor=661 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=306 (was 231) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=871 (was 2209)