2023-08-09 04:55:26,282 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce 2023-08-09 04:55:26,293 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-log-dir so I do NOT create it in target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f 2023-08-09 04:55:26,293 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-log-dir Erasing configuration value by system value. 2023-08-09 04:55:26,293 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-tmp-dir so I do NOT create it in target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f 2023-08-09 04:55:26,293 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-09 04:55:26,294 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f 2023-08-09 04:55:26,310 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-08-09 04:55:26,311 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL timeout: 13 mins 2023-08-09 04:55:26,345 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=266, ProcessCount=171, AvailableMemoryMB=1824 2023-08-09 04:55:26,356 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-log-dir so I do NOT create it in target/test-data/0443282d-4582-c0aa-6be0-03da9a438187 2023-08-09 04:55:26,356 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-log-dir Erasing configuration value by system value. 2023-08-09 04:55:26,357 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-tmp-dir so I do NOT create it in target/test-data/0443282d-4582-c0aa-6be0-03da9a438187 2023-08-09 04:55:26,357 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-09 04:55:26,357 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0443282d-4582-c0aa-6be0-03da9a438187 2023-08-09 04:55:26,364 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-log-dir so I do NOT create it in target/test-data/1ba70a3d-65a1-0320-d613-30f2c22a8f19 2023-08-09 04:55:26,364 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-log-dir Erasing configuration value by system value. 2023-08-09 04:55:26,364 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-tmp-dir so I do NOT create it in target/test-data/1ba70a3d-65a1-0320-d613-30f2c22a8f19 2023-08-09 04:55:26,364 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-09 04:55:26,365 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1ba70a3d-65a1-0320-d613-30f2c22a8f19 2023-08-09 04:55:26,368 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1, deleteOnExit=true 2023-08-09 04:55:26,484 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/zookeeper_0, clientPort=63198, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-09 04:55:26,499 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63198 2023-08-09 04:55:26,503 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-09 04:55:26,503 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-09 04:55:26,503 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/test.cache.data in system properties and HBase conf 2023-08-09 04:55:26,503 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop.tmp.dir in system properties and HBase conf 2023-08-09 04:55:26,504 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/hadoop.log.dir in system properties and HBase conf 2023-08-09 04:55:26,504 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-09 04:55:26,504 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-09 04:55:26,505 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-09 04:55:26,611 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-09 04:55:27,010 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-09 04:55:27,014 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-09 04:55:27,014 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-09 04:55:27,015 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-09 04:55:27,015 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-09 04:55:27,016 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-09 04:55:27,016 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-09 04:55:27,016 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-09 04:55:27,017 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-09 04:55:27,017 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-09 04:55:27,018 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/nfs.dump.dir in system properties and HBase conf 2023-08-09 04:55:27,018 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/java.io.tmpdir in system properties and HBase conf 2023-08-09 04:55:27,018 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-09 04:55:27,019 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-09 04:55:27,019 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-09 04:55:27,496 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-09 04:55:27,501 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-09 04:55:27,792 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-09 04:55:27,904 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-09 04:55:27,919 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:27,955 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:27,988 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/java.io.tmpdir/Jetty_localhost_43481_hdfs____i9794o/webapp 2023-08-09 04:55:28,123 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43481 2023-08-09 04:55:28,141 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-09 04:55:28,142 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-09 04:55:28,631 WARN [Listener at localhost/45825] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:28,694 WARN [Listener at localhost/45825] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-09 04:55:28,711 WARN [Listener at localhost/45825] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:28,717 INFO [Listener at localhost/45825] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:28,721 INFO [Listener at localhost/45825] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/java.io.tmpdir/Jetty_localhost_41525_datanode____hcbqpu/webapp 2023-08-09 04:55:28,817 INFO [Listener at localhost/45825] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41525 2023-08-09 04:55:29,135 WARN [Listener at localhost/36335] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:29,151 WARN [Listener at localhost/36335] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-09 04:55:29,155 WARN [Listener at localhost/36335] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:29,156 INFO [Listener at localhost/36335] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:29,163 INFO [Listener at localhost/36335] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/java.io.tmpdir/Jetty_localhost_44135_datanode____pzg5dm/webapp 2023-08-09 04:55:29,276 INFO [Listener at localhost/36335] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44135 2023-08-09 04:55:29,289 WARN [Listener at localhost/38459] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:29,592 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x648147702a8d8da7: Processing first storage report for DS-9193ffdc-8531-4e08-9163-ff0146654969 from datanode 9dc55359-3326-4192-b6d4-feee39bb2e6c 2023-08-09 04:55:29,593 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x648147702a8d8da7: from storage DS-9193ffdc-8531-4e08-9163-ff0146654969 node DatanodeRegistration(127.0.0.1:38063, datanodeUuid=9dc55359-3326-4192-b6d4-feee39bb2e6c, infoPort=34563, infoSecurePort=0, ipcPort=36335, storageInfo=lv=-57;cid=testClusterID;nsid=1833952195;c=1691556927574), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-09 04:55:29,593 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa55c7ebcd49e142b: Processing first storage report for DS-25f09e9e-adae-4cf2-bec8-46586d71f914 from datanode a3b0ca79-8cf0-4760-bcdf-7f10a751d92a 2023-08-09 04:55:29,594 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa55c7ebcd49e142b: from storage DS-25f09e9e-adae-4cf2-bec8-46586d71f914 node DatanodeRegistration(127.0.0.1:46353, datanodeUuid=a3b0ca79-8cf0-4760-bcdf-7f10a751d92a, infoPort=46829, infoSecurePort=0, ipcPort=38459, storageInfo=lv=-57;cid=testClusterID;nsid=1833952195;c=1691556927574), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:29,594 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x648147702a8d8da7: Processing first storage report for DS-b9139ecf-aa5d-44aa-9685-43dd5b7395e3 from datanode 9dc55359-3326-4192-b6d4-feee39bb2e6c 2023-08-09 04:55:29,594 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x648147702a8d8da7: from storage DS-b9139ecf-aa5d-44aa-9685-43dd5b7395e3 node DatanodeRegistration(127.0.0.1:38063, datanodeUuid=9dc55359-3326-4192-b6d4-feee39bb2e6c, infoPort=34563, infoSecurePort=0, ipcPort=36335, storageInfo=lv=-57;cid=testClusterID;nsid=1833952195;c=1691556927574), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:29,594 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa55c7ebcd49e142b: Processing first storage report for DS-26e99f36-96c8-43f7-98ff-818fd8277d7c from datanode a3b0ca79-8cf0-4760-bcdf-7f10a751d92a 2023-08-09 04:55:29,594 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa55c7ebcd49e142b: from storage DS-26e99f36-96c8-43f7-98ff-818fd8277d7c node DatanodeRegistration(127.0.0.1:46353, datanodeUuid=a3b0ca79-8cf0-4760-bcdf-7f10a751d92a, infoPort=46829, infoSecurePort=0, ipcPort=38459, storageInfo=lv=-57;cid=testClusterID;nsid=1833952195;c=1691556927574), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:29,672 DEBUG [Listener at localhost/38459] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce 2023-08-09 04:55:29,680 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:29,682 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:30,381 INFO [Listener at localhost/38459] util.FSUtils(471): Created version file at hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1 with version=8 2023-08-09 04:55:30,381 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/hbase-staging 2023-08-09 04:55:30,730 INFO [Listener at localhost/38459] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-09 04:55:31,237 INFO [Listener at localhost/38459] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-09 04:55:31,267 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,268 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,268 INFO [Listener at localhost/38459] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:31,268 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,268 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:31,406 INFO [Listener at localhost/38459] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:31,479 DEBUG [Listener at localhost/38459] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-09 04:55:31,586 INFO [Listener at localhost/38459] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38065 2023-08-09 04:55:31,596 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:31,598 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:31,619 INFO [Listener at localhost/38459] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38065 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:31,658 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:380650x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:31,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38065-0x101e814d6e10000 connected 2023-08-09 04:55:31,681 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:55:31,682 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:55:31,686 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-09 04:55:31,693 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38065 2023-08-09 04:55:31,693 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38065 2023-08-09 04:55:31,694 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38065 2023-08-09 04:55:31,695 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38065 2023-08-09 04:55:31,695 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38065 2023-08-09 04:55:31,699 INFO [Listener at localhost/38459] master.HMaster(444): hbase.rootdir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1, hbase.cluster.distributed=false 2023-08-09 04:55:31,791 INFO [Listener at localhost/38459] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-09 04:55:31,791 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,791 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,792 INFO [Listener at localhost/38459] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:31,792 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,792 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:31,797 INFO [Listener at localhost/38459] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:31,800 INFO [Listener at localhost/38459] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41117 2023-08-09 04:55:31,803 INFO [Listener at localhost/38459] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:31,809 DEBUG [Listener at localhost/38459] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:31,810 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:31,812 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:31,814 INFO [Listener at localhost/38459] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41117 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:31,818 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:411170x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:31,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41117-0x101e814d6e10001 connected 2023-08-09 04:55:31,819 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:55:31,821 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:55:31,822 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-09 04:55:31,823 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41117 2023-08-09 04:55:31,823 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41117 2023-08-09 04:55:31,824 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41117 2023-08-09 04:55:31,825 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41117 2023-08-09 04:55:31,825 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41117 2023-08-09 04:55:31,839 INFO [Listener at localhost/38459] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-09 04:55:31,839 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,839 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,840 INFO [Listener at localhost/38459] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:31,840 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:31,840 INFO [Listener at localhost/38459] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:31,841 INFO [Listener at localhost/38459] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:31,843 INFO [Listener at localhost/38459] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42251 2023-08-09 04:55:31,843 INFO [Listener at localhost/38459] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:31,846 DEBUG [Listener at localhost/38459] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:31,847 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:31,849 INFO [Listener at localhost/38459] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:31,850 INFO [Listener at localhost/38459] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42251 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:31,855 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:422510x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:31,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42251-0x101e814d6e10002 connected 2023-08-09 04:55:31,857 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:55:31,857 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:55:31,858 DEBUG [Listener at localhost/38459] zookeeper.ZKUtil(164): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-09 04:55:31,862 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42251 2023-08-09 04:55:31,862 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42251 2023-08-09 04:55:31,863 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42251 2023-08-09 04:55:31,863 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42251 2023-08-09 04:55:31,863 DEBUG [Listener at localhost/38459] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42251 2023-08-09 04:55:31,865 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:31,874 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-09 04:55:31,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:31,899 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-09 04:55:31,899 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-09 04:55:31,899 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-09 04:55:31,900 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:31,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-09 04:55:31,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,38065,1691556930543 from backup master directory 2023-08-09 04:55:31,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-09 04:55:31,909 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:31,909 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-09 04:55:31,910 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:31,910 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:31,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-09 04:55:31,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-09 04:55:32,033 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/hbase.id with ID: 2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 2023-08-09 04:55:32,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:32,112 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:32,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x22200881 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:32,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fea584c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:32,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:32,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-09 04:55:32,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:32,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:32,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store-tmp 2023-08-09 04:55:32,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:32,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:55:32,372 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:32,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:32,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:55:32,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:32,372 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:32,373 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:32,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:32,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:32,404 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/oldWALs, maxLogs=10 2023-08-09 04:55:32,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-09 04:55:32,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-09 04:55:32,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-09 04:55:32,490 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-09 04:55:32,493 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:55:32,493 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:55:32,504 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-09 04:55:32,569 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 2023-08-09 04:55:32,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK], DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK]] 2023-08-09 04:55:32,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:32,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:32,574 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:32,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:32,632 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:32,639 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-09 04:55:32,663 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-09 04:55:32,674 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:32,680 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:32,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:32,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:32,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:32,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68282253, jitterRate=0.017484858632087708}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:55:32,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:32,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-09 04:55:32,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-09 04:55:32,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-09 04:55:32,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-09 04:55:32,735 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-09 04:55:32,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-08-09 04:55:32,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-09 04:55:32,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-09 04:55:32,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-09 04:55:32,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-09 04:55:32,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-09 04:55:32,827 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-08-09 04:55:32,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-09 04:55:32,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-08-09 04:55:32,839 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:32,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-08-09 04:55:32,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-08-09 04:55:32,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-08-09 04:55:32,859 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-09 04:55:32,859 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-09 04:55:32,859 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-09 04:55:32,859 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:32,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38065,1691556930543, sessionid=0x101e814d6e10000, setting cluster-up flag (Was=false) 2023-08-09 04:55:32,878 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:32,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-08-09 04:55:32,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:32,891 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:32,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-08-09 04:55:32,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:32,901 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.hbase-snapshot/.tmp 2023-08-09 04:55:32,967 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(951): ClusterId : 2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 2023-08-09 04:55:32,967 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(951): ClusterId : 2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 2023-08-09 04:55:32,973 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:32,973 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:32,979 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:32,979 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:32,979 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:32,979 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:32,983 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:32,983 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:32,984 DEBUG [RS:0;jenkins-hbase4:41117] zookeeper.ReadOnlyZKClient(139): Connect 0x7129712f to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:32,985 DEBUG [RS:1;jenkins-hbase4:42251] zookeeper.ReadOnlyZKClient(139): Connect 0x04bb76aa to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:32,990 DEBUG [RS:1;jenkins-hbase4:42251] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a7c884d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:32,991 DEBUG [RS:0;jenkins-hbase4:41117] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7fee817a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:32,991 DEBUG [RS:1;jenkins-hbase4:42251] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e431dda, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:32,991 DEBUG [RS:0;jenkins-hbase4:41117] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bd11a5f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:33,013 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:42251 2023-08-09 04:55:33,015 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41117 2023-08-09 04:55:33,017 INFO [RS:1;jenkins-hbase4:42251] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:33,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:33,017 INFO [RS:1;jenkins-hbase4:42251] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:33,017 INFO [RS:0;jenkins-hbase4:41117] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:33,018 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:33,018 INFO [RS:0;jenkins-hbase4:41117] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:33,018 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:33,020 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38065,1691556930543 with isa=jenkins-hbase4.apache.org/172.31.14.131:42251, startcode=1691556931838 2023-08-09 04:55:33,020 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38065,1691556930543 with isa=jenkins-hbase4.apache.org/172.31.14.131:41117, startcode=1691556931790 2023-08-09 04:55:33,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:33,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:33,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:33,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:33,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-09 04:55:33,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:33,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691556963030 2023-08-09 04:55:33,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-09 04:55:33,037 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:33,037 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-09 04:55:33,040 DEBUG [RS:1;jenkins-hbase4:42251] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:33,040 DEBUG [RS:0;jenkins-hbase4:41117] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:33,043 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:33,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-09 04:55:33,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-09 04:55:33,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-09 04:55:33,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-09 04:55:33,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-09 04:55:33,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-09 04:55:33,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-09 04:55:33,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-09 04:55:33,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-09 04:55:33,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-09 04:55:33,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556933060,5,FailOnTimeoutGroup] 2023-08-09 04:55:33,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556933061,5,FailOnTimeoutGroup] 2023-08-09 04:55:33,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-09 04:55:33,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,065 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,082 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:33,084 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:33,084 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1 2023-08-09 04:55:33,106 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:33,109 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:55:33,111 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43641, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:33,111 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52403, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:33,112 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info 2023-08-09 04:55:33,113 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:55:33,114 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:33,115 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:55:33,117 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:33,118 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:55:33,119 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:33,119 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:55:33,122 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table 2023-08-09 04:55:33,123 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:55:33,123 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38065] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,124 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:33,126 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740 2023-08-09 04:55:33,127 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740 2023-08-09 04:55:33,131 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:55:33,133 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:55:33,136 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38065] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,139 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:33,140 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70842562, jitterRate=0.055636435747146606}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:55:33,140 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:55:33,140 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:55:33,140 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:55:33,140 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:55:33,140 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:55:33,141 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:55:33,141 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:55:33,142 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:55:33,143 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1 2023-08-09 04:55:33,143 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1 2023-08-09 04:55:33,143 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45825 2023-08-09 04:55:33,143 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45825 2023-08-09 04:55:33,143 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:33,143 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:33,148 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:33,148 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-09 04:55:33,150 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:55:33,151 DEBUG [RS:0;jenkins-hbase4:41117] zookeeper.ZKUtil(162): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,151 WARN [RS:0;jenkins-hbase4:41117] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:33,152 INFO [RS:0;jenkins-hbase4:41117] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:33,153 DEBUG [RS:1;jenkins-hbase4:42251] zookeeper.ZKUtil(162): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,153 INFO [RS:0;jenkins-hbase4:41117] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:33,153 WARN [RS:1;jenkins-hbase4:42251] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:33,153 INFO [RS:1;jenkins-hbase4:42251] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:33,153 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,153 INFO [RS:1;jenkins-hbase4:42251] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:33,154 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:33,154 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,154 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41117,1691556931790] 2023-08-09 04:55:33,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-09 04:55:33,165 DEBUG [RS:1;jenkins-hbase4:42251] zookeeper.ZKUtil(162): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,165 DEBUG [RS:0;jenkins-hbase4:41117] zookeeper.ZKUtil(162): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,165 DEBUG [RS:1;jenkins-hbase4:42251] zookeeper.ZKUtil(162): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,165 DEBUG [RS:0;jenkins-hbase4:41117] zookeeper.ZKUtil(162): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,174 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-09 04:55:33,177 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-09 04:55:33,177 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:33,177 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:33,186 INFO [RS:1;jenkins-hbase4:42251] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:33,186 INFO [RS:0;jenkins-hbase4:41117] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:33,207 INFO [RS:1;jenkins-hbase4:42251] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:33,207 INFO [RS:0;jenkins-hbase4:41117] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:33,211 INFO [RS:0;jenkins-hbase4:41117] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:33,211 INFO [RS:1;jenkins-hbase4:42251] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:33,212 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,212 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,212 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:33,213 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:33,220 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,220 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,220 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,220 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,221 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,222 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,222 DEBUG [RS:0;jenkins-hbase4:41117] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,222 DEBUG [RS:1;jenkins-hbase4:42251] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:33,223 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,223 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,223 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,226 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,227 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,227 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,241 INFO [RS:0;jenkins-hbase4:41117] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:33,241 INFO [RS:1;jenkins-hbase4:42251] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:33,244 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41117,1691556931790-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,244 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42251,1691556931838-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,261 INFO [RS:0;jenkins-hbase4:41117] regionserver.Replication(203): jenkins-hbase4.apache.org,41117,1691556931790 started 2023-08-09 04:55:33,261 INFO [RS:1;jenkins-hbase4:42251] regionserver.Replication(203): jenkins-hbase4.apache.org,42251,1691556931838 started 2023-08-09 04:55:33,261 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41117,1691556931790, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41117, sessionid=0x101e814d6e10001 2023-08-09 04:55:33,261 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42251,1691556931838, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42251, sessionid=0x101e814d6e10002 2023-08-09 04:55:33,262 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:33,262 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:33,262 DEBUG [RS:1;jenkins-hbase4:42251] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,262 DEBUG [RS:0;jenkins-hbase4:41117] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,262 DEBUG [RS:1;jenkins-hbase4:42251] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42251,1691556931838' 2023-08-09 04:55:33,262 DEBUG [RS:0;jenkins-hbase4:41117] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41117,1691556931790' 2023-08-09 04:55:33,263 DEBUG [RS:0;jenkins-hbase4:41117] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-09 04:55:33,263 DEBUG [RS:1;jenkins-hbase4:42251] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-09 04:55:33,264 DEBUG [RS:1;jenkins-hbase4:42251] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-09 04:55:33,264 DEBUG [RS:0;jenkins-hbase4:41117] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-09 04:55:33,264 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:33,264 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:33,264 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:33,264 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:33,264 DEBUG [RS:1;jenkins-hbase4:42251] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,264 DEBUG [RS:0;jenkins-hbase4:41117] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:33,265 DEBUG [RS:0;jenkins-hbase4:41117] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41117,1691556931790' 2023-08-09 04:55:33,265 DEBUG [RS:0;jenkins-hbase4:41117] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-09 04:55:33,265 DEBUG [RS:1;jenkins-hbase4:42251] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42251,1691556931838' 2023-08-09 04:55:33,265 DEBUG [RS:1;jenkins-hbase4:42251] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-09 04:55:33,266 DEBUG [RS:0;jenkins-hbase4:41117] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-09 04:55:33,266 DEBUG [RS:1;jenkins-hbase4:42251] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-09 04:55:33,266 DEBUG [RS:0;jenkins-hbase4:41117] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:33,266 INFO [RS:0;jenkins-hbase4:41117] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:33,266 DEBUG [RS:1;jenkins-hbase4:42251] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:33,266 INFO [RS:0;jenkins-hbase4:41117] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:33,266 INFO [RS:1;jenkins-hbase4:42251] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:33,267 INFO [RS:1;jenkins-hbase4:42251] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:33,329 DEBUG [jenkins-hbase4:38065] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-08-09 04:55:33,333 DEBUG [jenkins-hbase4:38065] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:33,340 DEBUG [jenkins-hbase4:38065] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:33,340 DEBUG [jenkins-hbase4:38065] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:33,340 DEBUG [jenkins-hbase4:38065] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:33,343 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42251,1691556931838, state=OPENING 2023-08-09 04:55:33,355 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-09 04:55:33,356 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:33,357 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-09 04:55:33,361 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42251,1691556931838}] 2023-08-09 04:55:33,377 INFO [RS:0;jenkins-hbase4:41117] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:33,377 INFO [RS:1;jenkins-hbase4:42251] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:33,381 INFO [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:55:33,382 INFO [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:55:33,405 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:55:33,405 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:55:33,407 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:55:33,407 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:55:33,413 INFO [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385 2023-08-09 04:55:33,413 INFO [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385 2023-08-09 04:55:33,413 DEBUG [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK], DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK]] 2023-08-09 04:55:33,414 DEBUG [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK], DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK]] 2023-08-09 04:55:33,550 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:33,553 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:33,557 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60898, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:33,567 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-09 04:55:33,568 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:33,568 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:33,568 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:33,572 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42251%2C1691556931838.meta, suffix=.meta, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:55:33,588 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:55:33,589 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:55:33,596 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.meta.1691556933573.meta 2023-08-09 04:55:33,603 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK], DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK]] 2023-08-09 04:55:33,603 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:33,605 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-09 04:55:33,629 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-09 04:55:33,635 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-09 04:55:33,644 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-09 04:55:33,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:33,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-09 04:55:33,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-09 04:55:33,648 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:55:33,650 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info 2023-08-09 04:55:33,650 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info 2023-08-09 04:55:33,651 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:55:33,652 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:33,652 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:55:33,654 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:33,654 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:33,655 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:55:33,656 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:33,656 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:55:33,658 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table 2023-08-09 04:55:33,658 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table 2023-08-09 04:55:33,658 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:55:33,659 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:33,661 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740 2023-08-09 04:55:33,664 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740 2023-08-09 04:55:33,667 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:55:33,670 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:55:33,671 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65877840, jitterRate=-0.018343687057495117}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:55:33,671 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:55:33,681 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691556933543 2023-08-09 04:55:33,700 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-09 04:55:33,701 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-09 04:55:33,701 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42251,1691556931838, state=OPEN 2023-08-09 04:55:33,704 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-09 04:55:33,704 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-09 04:55:33,709 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-09 04:55:33,709 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42251,1691556931838 in 343 msec 2023-08-09 04:55:33,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-09 04:55:33,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 548 msec 2023-08-09 04:55:33,719 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 774 msec 2023-08-09 04:55:33,719 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691556933719, completionTime=-1 2023-08-09 04:55:33,719 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-08-09 04:55:33,720 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-09 04:55:33,783 DEBUG [hconnection-0x78f05bde-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:33,788 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60910, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:33,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-08-09 04:55:33,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691556993821 2023-08-09 04:55:33,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691557053821 2023-08-09 04:55:33,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 101 msec 2023-08-09 04:55:33,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38065,1691556930543-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38065,1691556930543-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38065,1691556930543-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38065, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,847 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:33,853 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-09 04:55:33,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-09 04:55:33,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:33,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-09 04:55:33,875 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-09 04:55:33,877 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-09 04:55:33,897 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:33,900 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 empty. 2023-08-09 04:55:33,901 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:33,901 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-09 04:55:33,925 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:33,928 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => cb1d40dc2c341948bac0897d809dc4a0, NAME => 'hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp 2023-08-09 04:55:33,945 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:33,945 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing cb1d40dc2c341948bac0897d809dc4a0, disabling compactions & flushes 2023-08-09 04:55:33,945 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:33,945 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:33,945 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. after waiting 0 ms 2023-08-09 04:55:33,945 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:33,945 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:33,945 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for cb1d40dc2c341948bac0897d809dc4a0: 2023-08-09 04:55:33,950 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-09 04:55:33,969 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556933953"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691556933953"}]},"ts":"1691556933953"} 2023-08-09 04:55:33,995 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-09 04:55:33,997 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-09 04:55:34,001 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556933997"}]},"ts":"1691556933997"} 2023-08-09 04:55:34,004 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-09 04:55:34,008 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:34,010 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:34,010 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:34,010 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:34,013 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN}] 2023-08-09 04:55:34,015 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN 2023-08-09 04:55:34,017 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41117,1691556931790; forceNewPlan=false, retain=false 2023-08-09 04:55:34,169 INFO [jenkins-hbase4:38065] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-09 04:55:34,170 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=cb1d40dc2c341948bac0897d809dc4a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:34,171 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556934170"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556934170"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556934170"}]},"ts":"1691556934170"} 2023-08-09 04:55:34,176 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure cb1d40dc2c341948bac0897d809dc4a0, server=jenkins-hbase4.apache.org,41117,1691556931790}] 2023-08-09 04:55:34,330 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:34,331 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:34,334 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53780, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:34,341 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:34,341 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:34,345 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:55:34,363 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:55:34,363 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:55:34,369 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 2023-08-09 04:55:34,370 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK], DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK]] 2023-08-09 04:55:34,370 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cb1d40dc2c341948bac0897d809dc4a0, NAME => 'hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:34,371 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,371 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:34,371 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,372 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,374 INFO [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,377 DEBUG [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info 2023-08-09 04:55:34,377 DEBUG [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info 2023-08-09 04:55:34,378 INFO [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb1d40dc2c341948bac0897d809dc4a0 columnFamilyName info 2023-08-09 04:55:34,379 INFO [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] regionserver.HStore(310): Store=cb1d40dc2c341948bac0897d809dc4a0/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:34,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,381 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:34,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:34,391 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cb1d40dc2c341948bac0897d809dc4a0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72107563, jitterRate=0.07448641955852509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:55:34,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cb1d40dc2c341948bac0897d809dc4a0: 2023-08-09 04:55:34,393 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0., pid=6, masterSystemTime=1691556934330 2023-08-09 04:55:34,399 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:34,399 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:34,401 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=cb1d40dc2c341948bac0897d809dc4a0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:34,401 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556934400"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556934400"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556934400"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556934400"}]},"ts":"1691556934400"} 2023-08-09 04:55:34,409 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-09 04:55:34,409 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure cb1d40dc2c341948bac0897d809dc4a0, server=jenkins-hbase4.apache.org,41117,1691556931790 in 229 msec 2023-08-09 04:55:34,412 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-09 04:55:34,412 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN in 396 msec 2023-08-09 04:55:34,413 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-09 04:55:34,414 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556934414"}]},"ts":"1691556934414"} 2023-08-09 04:55:34,417 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-09 04:55:34,420 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-09 04:55:34,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 555 msec 2023-08-09 04:55:34,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-08-09 04:55:34,480 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-08-09 04:55:34,480 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:34,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:34,504 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53796, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:34,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-09 04:55:34,534 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-08-09 04:55:34,539 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 30 msec 2023-08-09 04:55:34,549 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-09 04:55:34,560 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-08-09 04:55:34,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-08-09 04:55:34,575 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-08-09 04:55:34,578 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-08-09 04:55:34,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.668sec 2023-08-09 04:55:34,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-09 04:55:34,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-09 04:55:34,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-09 04:55:34,584 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38065,1691556930543-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-09 04:55:34,585 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38065,1691556930543-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-09 04:55:34,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-09 04:55:34,674 DEBUG [Listener at localhost/38459] zookeeper.ReadOnlyZKClient(139): Connect 0x512eb72d to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:34,678 DEBUG [Listener at localhost/38459] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55d04483, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:34,693 DEBUG [hconnection-0x2b5a692-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:34,705 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60914, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:34,715 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:34,715 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-09 04:55:34,716 INFO [Listener at localhost/38459] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67, deleteOnExit=true 2023-08-09 04:55:34,716 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-09 04:55:34,727 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/test.cache.data in system properties and HBase conf 2023-08-09 04:55:34,727 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/hadoop.tmp.dir in system properties and HBase conf 2023-08-09 04:55:34,728 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/hadoop.log.dir in system properties and HBase conf 2023-08-09 04:55:34,728 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-09 04:55:34,728 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-09 04:55:34,728 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-09 04:55:34,728 DEBUG [Listener at localhost/38459] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-09 04:55:34,728 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-09 04:55:34,729 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-09 04:55:34,730 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-09 04:55:34,730 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/nfs.dump.dir in system properties and HBase conf 2023-08-09 04:55:34,730 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/java.io.tmpdir in system properties and HBase conf 2023-08-09 04:55:34,730 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-09 04:55:34,730 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-09 04:55:34,730 INFO [Listener at localhost/38459] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-09 04:55:34,736 WARN [Listener at localhost/38459] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-09 04:55:34,737 WARN [Listener at localhost/38459] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-09 04:55:34,784 WARN [Listener at localhost/38459] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:34,786 INFO [Listener at localhost/38459] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:34,791 INFO [Listener at localhost/38459] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/java.io.tmpdir/Jetty_localhost_33857_hdfs____ir89pg/webapp 2023-08-09 04:55:34,901 INFO [Listener at localhost/38459] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33857 2023-08-09 04:55:34,912 WARN [Listener at localhost/38459] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-09 04:55:34,912 WARN [Listener at localhost/38459] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-09 04:55:34,968 WARN [Listener at localhost/39381] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:34,984 WARN [Listener at localhost/39381] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-09 04:55:34,987 WARN [Listener at localhost/39381] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:34,988 INFO [Listener at localhost/39381] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:34,994 INFO [Listener at localhost/39381] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/java.io.tmpdir/Jetty_localhost_34993_datanode____kjchkp/webapp 2023-08-09 04:55:35,094 INFO [Listener at localhost/39381] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34993 2023-08-09 04:55:35,104 WARN [Listener at localhost/38339] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:35,123 WARN [Listener at localhost/38339] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-09 04:55:35,127 WARN [Listener at localhost/38339] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:35,129 INFO [Listener at localhost/38339] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:35,135 INFO [Listener at localhost/38339] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/java.io.tmpdir/Jetty_localhost_34683_datanode____p4s7tj/webapp 2023-08-09 04:55:35,224 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x74c2ddd63711d65f: Processing first storage report for DS-1ac82927-5264-4da4-9391-616e449c5539 from datanode fe09e2dd-113c-4ad4-80f4-36ed321deac3 2023-08-09 04:55:35,224 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x74c2ddd63711d65f: from storage DS-1ac82927-5264-4da4-9391-616e449c5539 node DatanodeRegistration(127.0.0.1:39401, datanodeUuid=fe09e2dd-113c-4ad4-80f4-36ed321deac3, infoPort=36819, infoSecurePort=0, ipcPort=38339, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,224 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x74c2ddd63711d65f: Processing first storage report for DS-435f42de-7a85-4521-b811-3282e52c324b from datanode fe09e2dd-113c-4ad4-80f4-36ed321deac3 2023-08-09 04:55:35,224 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x74c2ddd63711d65f: from storage DS-435f42de-7a85-4521-b811-3282e52c324b node DatanodeRegistration(127.0.0.1:39401, datanodeUuid=fe09e2dd-113c-4ad4-80f4-36ed321deac3, infoPort=36819, infoSecurePort=0, ipcPort=38339, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,253 INFO [Listener at localhost/38339] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34683 2023-08-09 04:55:35,263 WARN [Listener at localhost/38913] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:35,282 WARN [Listener at localhost/38913] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-09 04:55:35,285 WARN [Listener at localhost/38913] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:35,286 INFO [Listener at localhost/38913] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:35,292 INFO [Listener at localhost/38913] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/java.io.tmpdir/Jetty_localhost_38635_datanode____ngk8ai/webapp 2023-08-09 04:55:35,436 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8b1d8b641944ef03: Processing first storage report for DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835 from datanode 35ac5fa7-7302-40f8-b02c-58995eae4684 2023-08-09 04:55:35,436 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8b1d8b641944ef03: from storage DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835 node DatanodeRegistration(127.0.0.1:37263, datanodeUuid=35ac5fa7-7302-40f8-b02c-58995eae4684, infoPort=44671, infoSecurePort=0, ipcPort=38913, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,436 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8b1d8b641944ef03: Processing first storage report for DS-1cc97550-037a-4747-9904-8dbefd83325c from datanode 35ac5fa7-7302-40f8-b02c-58995eae4684 2023-08-09 04:55:35,436 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8b1d8b641944ef03: from storage DS-1cc97550-037a-4747-9904-8dbefd83325c node DatanodeRegistration(127.0.0.1:37263, datanodeUuid=35ac5fa7-7302-40f8-b02c-58995eae4684, infoPort=44671, infoSecurePort=0, ipcPort=38913, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,458 INFO [Listener at localhost/38913] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38635 2023-08-09 04:55:35,474 WARN [Listener at localhost/37213] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:35,500 WARN [Listener at localhost/37213] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-09 04:55:35,502 WARN [Listener at localhost/37213] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-09 04:55:35,503 INFO [Listener at localhost/37213] log.Slf4jLog(67): jetty-6.1.26 2023-08-09 04:55:35,510 INFO [Listener at localhost/37213] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/java.io.tmpdir/Jetty_localhost_35619_datanode____.oxf3dp/webapp 2023-08-09 04:55:35,581 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x32b0cd0c87076bc8: Processing first storage report for DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617 from datanode bc406285-cfbb-4280-a579-3ea083d46ea8 2023-08-09 04:55:35,581 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x32b0cd0c87076bc8: from storage DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617 node DatanodeRegistration(127.0.0.1:40039, datanodeUuid=bc406285-cfbb-4280-a579-3ea083d46ea8, infoPort=45691, infoSecurePort=0, ipcPort=37213, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,581 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x32b0cd0c87076bc8: Processing first storage report for DS-729af602-b490-4c63-abfc-82e94a1a5932 from datanode bc406285-cfbb-4280-a579-3ea083d46ea8 2023-08-09 04:55:35,581 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x32b0cd0c87076bc8: from storage DS-729af602-b490-4c63-abfc-82e94a1a5932 node DatanodeRegistration(127.0.0.1:40039, datanodeUuid=bc406285-cfbb-4280-a579-3ea083d46ea8, infoPort=45691, infoSecurePort=0, ipcPort=37213, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,619 INFO [Listener at localhost/37213] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35619 2023-08-09 04:55:35,627 WARN [Listener at localhost/38001] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-09 04:55:35,731 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4e1692150d01dffc: Processing first storage report for DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf from datanode a9d3f0a9-7c3a-4851-82ac-b181ddc8a2c6 2023-08-09 04:55:35,731 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4e1692150d01dffc: from storage DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf node DatanodeRegistration(127.0.0.1:38281, datanodeUuid=a9d3f0a9-7c3a-4851-82ac-b181ddc8a2c6, infoPort=36975, infoSecurePort=0, ipcPort=38001, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,731 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4e1692150d01dffc: Processing first storage report for DS-0c07c5fe-8699-49a3-bebc-7a8362b8bdd8 from datanode a9d3f0a9-7c3a-4851-82ac-b181ddc8a2c6 2023-08-09 04:55:35,731 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4e1692150d01dffc: from storage DS-0c07c5fe-8699-49a3-bebc-7a8362b8bdd8 node DatanodeRegistration(127.0.0.1:38281, datanodeUuid=a9d3f0a9-7c3a-4851-82ac-b181ddc8a2c6, infoPort=36975, infoSecurePort=0, ipcPort=38001, storageInfo=lv=-57;cid=testClusterID;nsid=505665514;c=1691556934740), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-09 04:55:35,738 DEBUG [Listener at localhost/38001] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f 2023-08-09 04:55:35,738 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,739 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,775 INFO [Listener at localhost/38001] util.FSUtils(471): Created version file at hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 with version=8 2023-08-09 04:55:35,775 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/hbase-staging 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:35,778 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:35,780 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38949 2023-08-09 04:55:35,780 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,782 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,783 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38949 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:35,788 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:389490x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:35,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38949-0x101e814d6e10007 connected 2023-08-09 04:55:35,800 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:35,800 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:35,801 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:35,801 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38949 2023-08-09 04:55:35,802 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38949 2023-08-09 04:55:35,802 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38949 2023-08-09 04:55:35,802 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38949 2023-08-09 04:55:35,802 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38949 2023-08-09 04:55:35,803 INFO [Listener at localhost/38001] master.HMaster(444): hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027, hbase.cluster.distributed=false 2023-08-09 04:55:35,817 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:35,818 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,818 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,818 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:35,818 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,818 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:35,818 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:35,820 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40127 2023-08-09 04:55:35,821 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:35,821 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:35,822 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,824 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,825 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40127 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:35,829 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:401270x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:35,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40127-0x101e814d6e10008 connected 2023-08-09 04:55:35,831 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:35,832 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:35,832 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:35,833 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40127 2023-08-09 04:55:35,833 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40127 2023-08-09 04:55:35,833 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40127 2023-08-09 04:55:35,834 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40127 2023-08-09 04:55:35,834 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40127 2023-08-09 04:55:35,846 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:35,847 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,847 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,847 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:35,847 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,847 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:35,847 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:35,849 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33953 2023-08-09 04:55:35,850 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:35,851 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:35,852 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,853 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,854 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33953 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:35,857 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:339530x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33953-0x101e814d6e10009 connected 2023-08-09 04:55:35,859 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:35,860 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:35,861 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:35,861 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33953 2023-08-09 04:55:35,862 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33953 2023-08-09 04:55:35,862 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33953 2023-08-09 04:55:35,867 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33953 2023-08-09 04:55:35,867 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33953 2023-08-09 04:55:35,882 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:35,882 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,883 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,883 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:35,883 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,883 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:35,883 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:35,886 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46479 2023-08-09 04:55:35,886 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:35,887 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:35,888 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,889 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,890 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46479 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:35,894 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:464790x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:35,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46479-0x101e814d6e1000a connected 2023-08-09 04:55:35,896 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:35,896 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:35,897 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:35,897 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46479 2023-08-09 04:55:35,897 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46479 2023-08-09 04:55:35,898 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46479 2023-08-09 04:55:35,898 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46479 2023-08-09 04:55:35,898 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46479 2023-08-09 04:55:35,911 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:35,911 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,912 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,912 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:35,912 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:35,912 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:35,912 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:35,914 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42521 2023-08-09 04:55:35,914 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:35,917 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:35,917 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,918 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,919 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42521 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:35,922 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:425210x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:35,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42521-0x101e814d6e1000b connected 2023-08-09 04:55:35,924 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:35,925 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:35,926 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:35,926 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42521 2023-08-09 04:55:35,926 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42521 2023-08-09 04:55:35,926 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42521 2023-08-09 04:55:35,927 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42521 2023-08-09 04:55:35,927 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42521 2023-08-09 04:55:35,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:35,932 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-09 04:55:35,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:35,934 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:35,934 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:35,935 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:35,936 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:35,936 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:35,937 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:35,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-09 04:55:35,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,38949,1691556935777 from backup master directory 2023-08-09 04:55:35,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-09 04:55:35,943 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:35,943 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-09 04:55:35,943 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:35,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:35,976 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/hbase.id with ID: 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:35,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:35,998 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:36,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2cbb9e7e to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:36,051 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@627ddfdd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:36,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:36,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-09 04:55:36,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:36,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:36,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store-tmp 2023-08-09 04:55:36,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:36,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:55:36,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:36,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:36,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:55:36,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:36,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:36,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:36,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:36,084 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:36,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/oldWALs, maxLogs=10 2023-08-09 04:55:36,108 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:36,111 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:36,111 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:36,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 2023-08-09 04:55:36,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK]] 2023-08-09 04:55:36,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:36,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:36,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:36,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:36,123 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:36,126 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-09 04:55:36,126 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-09 04:55:36,127 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:36,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:36,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:36,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:36,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74190759, jitterRate=0.10552845895290375}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:55:36,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:36,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-09 04:55:36,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-09 04:55:36,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-09 04:55:36,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-09 04:55:36,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-09 04:55:36,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 1 msec 2023-08-09 04:55:36,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-09 04:55:36,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-09 04:55:36,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-09 04:55:36,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-09 04:55:36,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-09 04:55:36,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-09 04:55:36,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-09 04:55:36,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-09 04:55:36,159 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:36,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-09 04:55:36,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-09 04:55:36,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-09 04:55:36,164 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:36,164 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:36,164 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:36,164 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:36,164 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:36,164 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:36,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38949,1691556935777, sessionid=0x101e814d6e10007, setting cluster-up flag (Was=false) 2023-08-09 04:55:36,171 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:36,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-09 04:55:36,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:36,185 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:36,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-09 04:55:36,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:36,196 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.hbase-snapshot/.tmp 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:36,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691556966202 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-09 04:55:36,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,204 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:36,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-09 04:55:36,204 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-09 04:55:36,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-09 04:55:36,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-09 04:55:36,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-09 04:55:36,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-09 04:55:36,206 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:36,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556936205,5,FailOnTimeoutGroup] 2023-08-09 04:55:36,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556936207,5,FailOnTimeoutGroup] 2023-08-09 04:55:36,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-09 04:55:36,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,232 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(951): ClusterId : 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:36,233 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:36,233 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(951): ClusterId : 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:36,234 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(951): ClusterId : 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:36,235 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(951): ClusterId : 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:36,235 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:36,237 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:36,236 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:36,239 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:36,239 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:36,240 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:36,239 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:36,240 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:36,240 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:36,240 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:55:36,243 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:36,243 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:36,243 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:36,245 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:36,248 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:36,248 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:36,250 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ReadOnlyZKClient(139): Connect 0x57b7b3e7 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:36,250 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ReadOnlyZKClient(139): Connect 0x3c1428f5 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:36,251 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:36,251 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:36,257 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ReadOnlyZKClient(139): Connect 0x7bb110ac to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:36,258 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ReadOnlyZKClient(139): Connect 0x7a502592 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:36,274 DEBUG [RS:1;jenkins-hbase4:33953] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32d3c107, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:36,276 DEBUG [RS:1;jenkins-hbase4:33953] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36ec4385, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:36,276 DEBUG [RS:0;jenkins-hbase4:40127] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5d4fd897, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:36,276 DEBUG [RS:0;jenkins-hbase4:40127] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@184d4d5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:36,289 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:33953 2023-08-09 04:55:36,289 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40127 2023-08-09 04:55:36,289 INFO [RS:0;jenkins-hbase4:40127] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:36,289 INFO [RS:0;jenkins-hbase4:40127] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:36,289 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:36,289 INFO [RS:1;jenkins-hbase4:33953] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:36,290 INFO [RS:1;jenkins-hbase4:33953] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:36,290 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:36,290 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38949,1691556935777 with isa=jenkins-hbase4.apache.org/172.31.14.131:40127, startcode=1691556935817 2023-08-09 04:55:36,290 DEBUG [RS:3;jenkins-hbase4:42521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7497d4a8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:36,290 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38949,1691556935777 with isa=jenkins-hbase4.apache.org/172.31.14.131:33953, startcode=1691556935846 2023-08-09 04:55:36,291 DEBUG [RS:0;jenkins-hbase4:40127] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:36,291 DEBUG [RS:1;jenkins-hbase4:33953] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:36,291 DEBUG [RS:3;jenkins-hbase4:42521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70a84ee2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:36,292 DEBUG [RS:2;jenkins-hbase4:46479] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f8e8b02, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:36,292 DEBUG [RS:2;jenkins-hbase4:46479] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a8c1a97, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:36,304 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:46479 2023-08-09 04:55:36,304 INFO [RS:2;jenkins-hbase4:46479] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:36,304 INFO [RS:2;jenkins-hbase4:46479] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:36,304 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:36,305 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38949,1691556935777 with isa=jenkins-hbase4.apache.org/172.31.14.131:46479, startcode=1691556935882 2023-08-09 04:55:36,305 DEBUG [RS:2;jenkins-hbase4:46479] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:36,309 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:42521 2023-08-09 04:55:36,309 INFO [RS:3;jenkins-hbase4:42521] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:36,309 INFO [RS:3;jenkins-hbase4:42521] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:36,309 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:36,310 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38949,1691556935777 with isa=jenkins-hbase4.apache.org/172.31.14.131:42521, startcode=1691556935911 2023-08-09 04:55:36,310 DEBUG [RS:3;jenkins-hbase4:42521] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:36,319 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50319, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:36,325 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49163, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:36,326 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38949] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,327 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48679, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:36,327 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38949] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,328 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48783, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:36,328 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:36,328 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38949] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,328 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38949] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,328 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:55:36,329 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39381 2023-08-09 04:55:36,329 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:55:36,329 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39381 2023-08-09 04:55:36,329 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:36,329 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:55:36,329 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39381 2023-08-09 04:55:36,329 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:36,329 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:36,329 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:55:36,329 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39381 2023-08-09 04:55:36,330 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:36,341 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:36,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:55:36,342 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ZKUtil(162): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,342 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ZKUtil(162): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,342 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ZKUtil(162): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,342 WARN [RS:1;jenkins-hbase4:33953] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:36,342 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ZKUtil(162): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,342 INFO [RS:1;jenkins-hbase4:33953] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:36,342 WARN [RS:0;jenkins-hbase4:40127] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:36,342 INFO [RS:1;jenkins-hbase4:33953] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:36,342 WARN [RS:2;jenkins-hbase4:46479] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:36,342 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,342 INFO [RS:2;jenkins-hbase4:46479] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:36,342 INFO [RS:0;jenkins-hbase4:40127] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:36,342 INFO [RS:2;jenkins-hbase4:46479] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:36,342 WARN [RS:3;jenkins-hbase4:42521] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:36,343 INFO [RS:0;jenkins-hbase4:40127] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:36,343 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,343 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,343 INFO [RS:3;jenkins-hbase4:42521] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:36,343 INFO [RS:3;jenkins-hbase4:42521] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:36,343 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,348 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40127,1691556935817] 2023-08-09 04:55:36,348 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42521,1691556935911] 2023-08-09 04:55:36,348 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33953,1691556935846] 2023-08-09 04:55:36,348 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46479,1691556935882] 2023-08-09 04:55:36,349 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:55:36,349 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:55:36,353 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,353 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:55:36,359 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ZKUtil(162): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,360 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ZKUtil(162): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,361 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ZKUtil(162): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,361 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ZKUtil(162): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,361 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:36,361 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ZKUtil(162): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,361 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ZKUtil(162): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,361 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:55:36,362 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ZKUtil(162): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,362 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ZKUtil(162): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,362 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ZKUtil(162): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,362 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ZKUtil(162): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,365 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ZKUtil(162): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,365 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ZKUtil(162): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,365 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ZKUtil(162): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,365 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ZKUtil(162): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,362 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,362 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ZKUtil(162): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,366 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:55:36,367 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:36,367 INFO [RS:2;jenkins-hbase4:46479] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:36,367 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:36,368 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ZKUtil(162): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,369 DEBUG [RS:1;jenkins-hbase4:33953] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:36,370 INFO [RS:1;jenkins-hbase4:33953] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:36,370 INFO [RS:3;jenkins-hbase4:42521] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:36,371 INFO [RS:2;jenkins-hbase4:46479] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:36,372 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:36,373 INFO [RS:0;jenkins-hbase4:40127] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:36,375 INFO [RS:2;jenkins-hbase4:46479] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:36,375 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,378 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:55:36,379 INFO [RS:1;jenkins-hbase4:33953] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:36,379 INFO [RS:3;jenkins-hbase4:42521] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:36,379 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:55:36,380 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,382 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:36,385 INFO [RS:1;jenkins-hbase4:33953] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:36,385 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,385 INFO [RS:3;jenkins-hbase4:42521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:36,385 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,385 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:36,386 INFO [RS:0;jenkins-hbase4:40127] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:36,386 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:55:36,386 INFO [RS:0;jenkins-hbase4:40127] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:36,386 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,387 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:55:36,391 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:55:36,393 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:55:36,400 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:36,401 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:36,403 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,403 DEBUG [RS:1;jenkins-hbase4:33953] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,409 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 DEBUG [RS:2;jenkins-hbase4:46479] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,410 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,411 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,411 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,411 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,412 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,412 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,414 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,414 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,414 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,414 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:36,415 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:36,415 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:0;jenkins-hbase4:40127] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,415 DEBUG [RS:3;jenkins-hbase4:42521] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:36,418 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:36,418 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,419 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,420 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,420 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72085319, jitterRate=0.07415495812892914}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:55:36,420 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:55:36,420 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:55:36,420 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:55:36,420 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:55:36,420 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:55:36,420 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:55:36,421 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,421 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,421 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,428 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:55:36,429 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:55:36,431 INFO [RS:1;jenkins-hbase4:33953] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:36,431 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33953,1691556935846-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,431 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:36,431 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-09 04:55:36,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-09 04:55:36,434 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-09 04:55:36,434 INFO [RS:2;jenkins-hbase4:46479] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:36,435 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46479,1691556935882-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,437 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-09 04:55:36,445 INFO [RS:0;jenkins-hbase4:40127] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:36,445 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40127,1691556935817-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,445 INFO [RS:3;jenkins-hbase4:42521] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:36,445 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42521,1691556935911-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,445 INFO [RS:1;jenkins-hbase4:33953] regionserver.Replication(203): jenkins-hbase4.apache.org,33953,1691556935846 started 2023-08-09 04:55:36,445 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33953,1691556935846, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33953, sessionid=0x101e814d6e10009 2023-08-09 04:55:36,446 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:36,446 DEBUG [RS:1;jenkins-hbase4:33953] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,447 DEBUG [RS:1;jenkins-hbase4:33953] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33953,1691556935846' 2023-08-09 04:55:36,447 DEBUG [RS:1;jenkins-hbase4:33953] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-09 04:55:36,447 DEBUG [RS:1;jenkins-hbase4:33953] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-09 04:55:36,448 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:36,448 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:36,449 DEBUG [RS:1;jenkins-hbase4:33953] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:36,449 DEBUG [RS:1;jenkins-hbase4:33953] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33953,1691556935846' 2023-08-09 04:55:36,449 DEBUG [RS:1;jenkins-hbase4:33953] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-09 04:55:36,449 DEBUG [RS:1;jenkins-hbase4:33953] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-09 04:55:36,450 DEBUG [RS:1;jenkins-hbase4:33953] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:36,450 INFO [RS:1;jenkins-hbase4:33953] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:36,450 INFO [RS:1;jenkins-hbase4:33953] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:36,454 INFO [RS:2;jenkins-hbase4:46479] regionserver.Replication(203): jenkins-hbase4.apache.org,46479,1691556935882 started 2023-08-09 04:55:36,454 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46479,1691556935882, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46479, sessionid=0x101e814d6e1000a 2023-08-09 04:55:36,455 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:36,455 DEBUG [RS:2;jenkins-hbase4:46479] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,455 DEBUG [RS:2;jenkins-hbase4:46479] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46479,1691556935882' 2023-08-09 04:55:36,455 DEBUG [RS:2;jenkins-hbase4:46479] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-09 04:55:36,455 DEBUG [RS:2;jenkins-hbase4:46479] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-09 04:55:36,456 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:36,456 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:36,456 DEBUG [RS:2;jenkins-hbase4:46479] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,456 DEBUG [RS:2;jenkins-hbase4:46479] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46479,1691556935882' 2023-08-09 04:55:36,456 DEBUG [RS:2;jenkins-hbase4:46479] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-09 04:55:36,456 DEBUG [RS:2;jenkins-hbase4:46479] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-09 04:55:36,457 DEBUG [RS:2;jenkins-hbase4:46479] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:36,457 INFO [RS:2;jenkins-hbase4:46479] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:36,457 INFO [RS:2;jenkins-hbase4:46479] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:36,458 INFO [RS:0;jenkins-hbase4:40127] regionserver.Replication(203): jenkins-hbase4.apache.org,40127,1691556935817 started 2023-08-09 04:55:36,458 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40127,1691556935817, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40127, sessionid=0x101e814d6e10008 2023-08-09 04:55:36,458 INFO [RS:3;jenkins-hbase4:42521] regionserver.Replication(203): jenkins-hbase4.apache.org,42521,1691556935911 started 2023-08-09 04:55:36,458 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:36,458 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42521,1691556935911, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42521, sessionid=0x101e814d6e1000b 2023-08-09 04:55:36,458 DEBUG [RS:0;jenkins-hbase4:40127] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,458 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:36,458 DEBUG [RS:3;jenkins-hbase4:42521] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,458 DEBUG [RS:3;jenkins-hbase4:42521] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42521,1691556935911' 2023-08-09 04:55:36,458 DEBUG [RS:3;jenkins-hbase4:42521] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-09 04:55:36,458 DEBUG [RS:0;jenkins-hbase4:40127] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40127,1691556935817' 2023-08-09 04:55:36,458 DEBUG [RS:0;jenkins-hbase4:40127] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-09 04:55:36,459 DEBUG [RS:3;jenkins-hbase4:42521] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-09 04:55:36,459 DEBUG [RS:0;jenkins-hbase4:40127] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-09 04:55:36,459 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:36,459 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:36,460 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:36,459 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:36,460 DEBUG [RS:3;jenkins-hbase4:42521] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:36,460 DEBUG [RS:3;jenkins-hbase4:42521] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42521,1691556935911' 2023-08-09 04:55:36,460 DEBUG [RS:3;jenkins-hbase4:42521] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-09 04:55:36,460 DEBUG [RS:0;jenkins-hbase4:40127] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:36,460 DEBUG [RS:0;jenkins-hbase4:40127] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40127,1691556935817' 2023-08-09 04:55:36,460 DEBUG [RS:0;jenkins-hbase4:40127] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-09 04:55:36,460 DEBUG [RS:3;jenkins-hbase4:42521] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-09 04:55:36,460 DEBUG [RS:0;jenkins-hbase4:40127] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-09 04:55:36,461 DEBUG [RS:3;jenkins-hbase4:42521] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:36,461 INFO [RS:3;jenkins-hbase4:42521] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:36,461 INFO [RS:3;jenkins-hbase4:42521] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:36,465 DEBUG [RS:0;jenkins-hbase4:40127] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:36,465 INFO [RS:0;jenkins-hbase4:40127] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:36,465 INFO [RS:0;jenkins-hbase4:40127] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:36,550 INFO [RS:1;jenkins-hbase4:33953] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:36,553 INFO [RS:1;jenkins-hbase4:33953] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33953%2C1691556935846.jenkins-hbase4.apache.org%2C33953%2C1691556935846.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:36,557 INFO [RS:2;jenkins-hbase4:46479] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:36,559 INFO [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46479%2C1691556935882.jenkins-hbase4.apache.org%2C46479%2C1691556935882.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:36,561 INFO [RS:3;jenkins-hbase4:42521] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:36,565 INFO [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42521%2C1691556935911.jenkins-hbase4.apache.org%2C42521%2C1691556935911.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:36,565 INFO [RS:0;jenkins-hbase4:40127] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:36,568 INFO [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40127%2C1691556935817.jenkins-hbase4.apache.org%2C40127%2C1691556935817.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:36,582 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:36,583 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:36,585 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:36,591 DEBUG [jenkins-hbase4:38949] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-08-09 04:55:36,592 DEBUG [jenkins-hbase4:38949] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:36,593 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:36,593 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:36,593 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:36,595 DEBUG [jenkins-hbase4:38949] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:36,595 DEBUG [jenkins-hbase4:38949] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:36,595 INFO [RS:1;jenkins-hbase4:33953] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846/jenkins-hbase4.apache.org%2C33953%2C1691556935846.jenkins-hbase4.apache.org%2C33953%2C1691556935846.regiongroup-0.1691556936554 2023-08-09 04:55:36,595 DEBUG [jenkins-hbase4:38949] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-09 04:55:36,595 DEBUG [jenkins-hbase4:38949] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-09 04:55:36,595 DEBUG [jenkins-hbase4:38949] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:36,595 DEBUG [RS:1;jenkins-hbase4:33953] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK]] 2023-08-09 04:55:36,597 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46479,1691556935882, state=OPENING 2023-08-09 04:55:36,602 INFO [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882/jenkins-hbase4.apache.org%2C46479%2C1691556935882.jenkins-hbase4.apache.org%2C46479%2C1691556935882.regiongroup-0.1691556936560 2023-08-09 04:55:36,603 DEBUG [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK]] 2023-08-09 04:55:36,603 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-09 04:55:36,610 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:36,611 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:36,611 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:36,611 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-09 04:55:36,616 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46479,1691556935882}] 2023-08-09 04:55:36,618 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:36,618 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:36,623 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:36,623 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:36,627 INFO [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911/jenkins-hbase4.apache.org%2C42521%2C1691556935911.jenkins-hbase4.apache.org%2C42521%2C1691556935911.regiongroup-0.1691556936566 2023-08-09 04:55:36,627 DEBUG [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK]] 2023-08-09 04:55:36,628 INFO [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817/jenkins-hbase4.apache.org%2C40127%2C1691556935817.jenkins-hbase4.apache.org%2C40127%2C1691556935817.regiongroup-0.1691556936570 2023-08-09 04:55:36,629 DEBUG [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK]] 2023-08-09 04:55:36,781 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:36,781 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:36,784 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53584, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:36,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-09 04:55:36,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:36,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:36,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:36,794 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46479%2C1691556935882.meta, suffix=.meta, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:36,817 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:36,818 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:36,819 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:36,835 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882/jenkins-hbase4.apache.org%2C46479%2C1691556935882.meta.1691556936794.meta 2023-08-09 04:55:36,840 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK]] 2023-08-09 04:55:36,841 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:36,842 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-09 04:55:36,842 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-09 04:55:36,846 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-09 04:55:36,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-09 04:55:36,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:36,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-09 04:55:36,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-09 04:55:36,849 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:55:36,851 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:55:36,851 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:55:36,852 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:55:36,853 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,854 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:55:36,855 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:36,856 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:36,856 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:55:36,858 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,858 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:55:36,859 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:55:36,860 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:55:36,860 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:55:36,861 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:36,863 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:55:36,866 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:55:36,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:55:36,876 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:55:36,878 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70422161, jitterRate=0.04937197268009186}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:55:36,878 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:55:36,880 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691556936781 2023-08-09 04:55:36,887 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-09 04:55:36,888 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-09 04:55:36,889 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46479,1691556935882, state=OPEN 2023-08-09 04:55:36,891 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-09 04:55:36,892 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-09 04:55:36,897 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-09 04:55:36,897 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46479,1691556935882 in 276 msec 2023-08-09 04:55:36,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-09 04:55:36,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 466 msec 2023-08-09 04:55:36,912 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 709 msec 2023-08-09 04:55:36,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691556936913, completionTime=-1 2023-08-09 04:55:36,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-08-09 04:55:36,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-09 04:55:36,920 DEBUG [hconnection-0x56b44e7c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:36,922 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53600, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:36,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-08-09 04:55:36,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691556996926 2023-08-09 04:55:36,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691557056926 2023-08-09 04:55:36,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 12 msec 2023-08-09 04:55:36,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38949,1691556935777-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38949,1691556935777-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38949,1691556935777-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38949, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:36,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-09 04:55:36,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:36,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-09 04:55:36,970 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-09 04:55:36,970 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-09 04:55:36,972 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-09 04:55:36,978 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:36,979 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 empty. 2023-08-09 04:55:36,979 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:36,979 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-09 04:55:37,078 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:37,080 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 36954931527a871a4ac33b21c04863f3, NAME => 'hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp 2023-08-09 04:55:37,144 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:37,144 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 36954931527a871a4ac33b21c04863f3, disabling compactions & flushes 2023-08-09 04:55:37,144 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,144 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,145 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. after waiting 0 ms 2023-08-09 04:55:37,145 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,145 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,145 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:55:37,152 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-09 04:55:37,156 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556937155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691556937155"}]},"ts":"1691556937155"} 2023-08-09 04:55:37,160 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-09 04:55:37,163 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-09 04:55:37,163 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556937163"}]},"ts":"1691556937163"} 2023-08-09 04:55:37,169 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-09 04:55:37,174 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:37,175 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:37,175 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:37,175 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-09 04:55:37,175 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-09 04:55:37,175 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:37,176 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN}] 2023-08-09 04:55:37,179 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN 2023-08-09 04:55:37,181 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42521,1691556935911; forceNewPlan=false, retain=false 2023-08-09 04:55:37,331 INFO [jenkins-hbase4:38949] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-09 04:55:37,333 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=36954931527a871a4ac33b21c04863f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:37,333 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556937333"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556937333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556937333"}]},"ts":"1691556937333"} 2023-08-09 04:55:37,336 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 36954931527a871a4ac33b21c04863f3, server=jenkins-hbase4.apache.org,42521,1691556935911}] 2023-08-09 04:55:37,491 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:37,492 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:37,494 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33064, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:37,500 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,500 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:37,503 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42521%2C1691556935911.jenkins-hbase4.apache.org%2C42521%2C1691556935911.regiongroup-1, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:37,523 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:37,526 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:37,526 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:37,536 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911/jenkins-hbase4.apache.org%2C42521%2C1691556935911.jenkins-hbase4.apache.org%2C42521%2C1691556935911.regiongroup-1.1691556937503 2023-08-09 04:55:37,537 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK]] 2023-08-09 04:55:37,537 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36954931527a871a4ac33b21c04863f3, NAME => 'hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:37,538 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,538 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:37,538 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,538 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,540 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,542 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info 2023-08-09 04:55:37,542 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info 2023-08-09 04:55:37,544 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36954931527a871a4ac33b21c04863f3 columnFamilyName info 2023-08-09 04:55:37,545 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(310): Store=36954931527a871a4ac33b21c04863f3/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:37,547 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,548 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,553 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:37,560 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:37,561 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36954931527a871a4ac33b21c04863f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63559616, jitterRate=-0.052887916564941406}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:55:37,561 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:55:37,563 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., pid=6, masterSystemTime=1691556937491 2023-08-09 04:55:37,568 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,569 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:37,570 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=36954931527a871a4ac33b21c04863f3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:37,571 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556937570"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556937570"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556937570"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556937570"}]},"ts":"1691556937570"} 2023-08-09 04:55:37,577 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-09 04:55:37,577 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 36954931527a871a4ac33b21c04863f3, server=jenkins-hbase4.apache.org,42521,1691556935911 in 238 msec 2023-08-09 04:55:37,581 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-09 04:55:37,582 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN in 401 msec 2023-08-09 04:55:37,583 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-09 04:55:37,583 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556937583"}]},"ts":"1691556937583"} 2023-08-09 04:55:37,585 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-09 04:55:37,588 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-09 04:55:37,591 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 632 msec 2023-08-09 04:55:37,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-08-09 04:55:37,671 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-08-09 04:55:37,672 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:37,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:37,676 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33076, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:37,680 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-09 04:55:37,693 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-08-09 04:55:37,697 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 17 msec 2023-08-09 04:55:37,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-09 04:55:37,712 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-08-09 04:55:37,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-08-09 04:55:37,727 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-09 04:55:37,730 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-09 04:55:37,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.787sec 2023-08-09 04:55:37,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-09 04:55:37,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-09 04:55:37,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-09 04:55:37,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38949,1691556935777-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-09 04:55:37,732 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38949,1691556935777-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-09 04:55:37,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-09 04:55:37,736 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x037ad110 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:37,742 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@48989e1b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:37,743 DEBUG [hconnection-0x55e24911-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:37,745 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53606, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:37,747 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:37,756 DEBUG [Listener at localhost/38001] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-09 04:55:37,758 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40082, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-09 04:55:37,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:37,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-08-09 04:55:37,774 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-09 04:55:37,776 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-09 04:55:37,779 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:37,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-08-09 04:55:37,780 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 empty. 2023-08-09 04:55:37,780 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:37,780 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-08-09 04:55:37,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:37,802 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:37,804 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4d1b412f72764eba378319d319bc7848, NAME => 't1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp 2023-08-09 04:55:37,816 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:37,816 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 4d1b412f72764eba378319d319bc7848, disabling compactions & flushes 2023-08-09 04:55:37,816 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:37,816 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:37,816 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. after waiting 0 ms 2023-08-09 04:55:37,816 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:37,816 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:37,816 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 4d1b412f72764eba378319d319bc7848: 2023-08-09 04:55:37,820 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-09 04:55:37,821 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556937821"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691556937821"}]},"ts":"1691556937821"} 2023-08-09 04:55:37,824 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-09 04:55:37,825 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-09 04:55:37,825 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556937825"}]},"ts":"1691556937825"} 2023-08-09 04:55:37,827 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-08-09 04:55:37,831 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:37,832 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:37,832 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:37,832 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:37,833 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN}] 2023-08-09 04:55:37,834 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN 2023-08-09 04:55:37,835 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42251,1691556931838; forceNewPlan=false, retain=false 2023-08-09 04:55:37,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:37,986 INFO [jenkins-hbase4:38065] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-09 04:55:37,988 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4d1b412f72764eba378319d319bc7848, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:37,988 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556937987"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556937987"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556937987"}]},"ts":"1691556937987"} 2023-08-09 04:55:37,991 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 4d1b412f72764eba378319d319bc7848, server=jenkins-hbase4.apache.org,42251,1691556931838}] 2023-08-09 04:55:38,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:38,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:38,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:38,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:55:38,168 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:55:38,169 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:55:38,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153 2023-08-09 04:55:38,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK], DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK]] 2023-08-09 04:55:38,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4d1b412f72764eba378319d319bc7848, NAME => 't1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:38,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:38,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,175 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,177 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1 2023-08-09 04:55:38,177 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1 2023-08-09 04:55:38,177 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4d1b412f72764eba378319d319bc7848 columnFamilyName cf1 2023-08-09 04:55:38,178 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(310): Store=4d1b412f72764eba378319d319bc7848/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:38,178 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,180 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep 2023-08-09 04:55:38,180 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep 2023-08-09 04:55:38,181 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4d1b412f72764eba378319d319bc7848 columnFamilyName norep 2023-08-09 04:55:38,181 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(310): Store=4d1b412f72764eba378319d319bc7848/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:38,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:55:38,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:38,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:38,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4d1b412f72764eba378319d319bc7848; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73162614, jitterRate=0.09020790457725525}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:55:38,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4d1b412f72764eba378319d319bc7848: 2023-08-09 04:55:38,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848., pid=11, masterSystemTime=1691556938143 2023-08-09 04:55:38,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:38,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:38,201 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4d1b412f72764eba378319d319bc7848, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:38,201 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556938201"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556938201"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556938201"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556938201"}]},"ts":"1691556938201"} 2023-08-09 04:55:38,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-09 04:55:38,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 4d1b412f72764eba378319d319bc7848, server=jenkins-hbase4.apache.org,42251,1691556931838 in 213 msec 2023-08-09 04:55:38,214 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-09 04:55:38,216 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN in 379 msec 2023-08-09 04:55:38,217 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-09 04:55:38,218 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556938217"}]},"ts":"1691556938217"} 2023-08-09 04:55:38,220 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-08-09 04:55:38,227 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-09 04:55:38,230 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 457 msec 2023-08-09 04:55:38,261 INFO [jenkins-hbase4:42251Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:38,261 INFO [jenkins-hbase4:41117Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:38,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:38,401 INFO [Listener at localhost/38001] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-08-09 04:55:38,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:38,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-08-09 04:55:38,407 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-09 04:55:38,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-08-09 04:55:38,409 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-09 04:55:38,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:38,412 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,412 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 empty. 2023-08-09 04:55:38,413 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,413 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-08-09 04:55:38,434 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:38,438 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2ccff9a5354735fa815894d8a555d598, NAME => 't2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.tmp 2023-08-09 04:55:38,464 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:38,464 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 2ccff9a5354735fa815894d8a555d598, disabling compactions & flushes 2023-08-09 04:55:38,464 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,464 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,464 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. after waiting 0 ms 2023-08-09 04:55:38,464 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,465 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,465 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 2ccff9a5354735fa815894d8a555d598: 2023-08-09 04:55:38,469 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-09 04:55:38,473 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556938473"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691556938473"}]},"ts":"1691556938473"} 2023-08-09 04:55:38,476 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-09 04:55:38,477 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-09 04:55:38,477 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556938477"}]},"ts":"1691556938477"} 2023-08-09 04:55:38,479 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-08-09 04:55:38,483 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:38,485 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:38,485 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:38,485 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:38,485 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN}] 2023-08-09 04:55:38,487 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN 2023-08-09 04:55:38,488 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42251,1691556931838; forceNewPlan=false, retain=false 2023-08-09 04:55:38,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:38,639 INFO [jenkins-hbase4:38065] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-09 04:55:38,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2ccff9a5354735fa815894d8a555d598, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:38,641 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556938640"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556938640"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556938640"}]},"ts":"1691556938640"} 2023-08-09 04:55:38,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 2ccff9a5354735fa815894d8a555d598, server=jenkins-hbase4.apache.org,42251,1691556931838}] 2023-08-09 04:55:38,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:38,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2ccff9a5354735fa815894d8a555d598, NAME => 't2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:38,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:38,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,805 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,808 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1 2023-08-09 04:55:38,808 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1 2023-08-09 04:55:38,809 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ccff9a5354735fa815894d8a555d598 columnFamilyName cf1 2023-08-09 04:55:38,810 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(310): Store=2ccff9a5354735fa815894d8a555d598/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:38,810 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,813 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep 2023-08-09 04:55:38,813 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep 2023-08-09 04:55:38,813 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ccff9a5354735fa815894d8a555d598 columnFamilyName norep 2023-08-09 04:55:38,814 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(310): Store=2ccff9a5354735fa815894d8a555d598/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:38,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:55:38,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:38,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:38,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2ccff9a5354735fa815894d8a555d598; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66463754, jitterRate=-0.009612888097763062}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:55:38,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2ccff9a5354735fa815894d8a555d598: 2023-08-09 04:55:38,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598., pid=14, masterSystemTime=1691556938796 2023-08-09 04:55:38,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:38,829 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2ccff9a5354735fa815894d8a555d598, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:38,829 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556938829"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556938829"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556938829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556938829"}]},"ts":"1691556938829"} 2023-08-09 04:55:38,833 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-09 04:55:38,834 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 2ccff9a5354735fa815894d8a555d598, server=jenkins-hbase4.apache.org,42251,1691556931838 in 188 msec 2023-08-09 04:55:38,836 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-09 04:55:38,836 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN in 349 msec 2023-08-09 04:55:38,837 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-09 04:55:38,837 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556938837"}]},"ts":"1691556938837"} 2023-08-09 04:55:38,839 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-08-09 04:55:38,841 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-09 04:55:38,844 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 438 msec 2023-08-09 04:55:39,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:39,014 INFO [Listener at localhost/38001] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-08-09 04:55:39,019 DEBUG [Listener at localhost/38001] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-09 04:55:39,021 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33810, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-09 04:55:39,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:39,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-08-09 04:55:39,028 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-09 04:55:39,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-08-09 04:55:39,029 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-09 04:55:39,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:39,032 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,032 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea empty. 2023-08-09 04:55:39,033 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,033 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-08-09 04:55:39,057 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:39,058 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => baf34b839cf96847099b70ccf7c3d7ea, NAME => 't1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp 2023-08-09 04:55:39,075 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:39,075 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing baf34b839cf96847099b70ccf7c3d7ea, disabling compactions & flushes 2023-08-09 04:55:39,075 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,075 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,075 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. after waiting 0 ms 2023-08-09 04:55:39,075 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,075 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,075 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:55:39,079 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-09 04:55:39,080 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556939080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691556939080"}]},"ts":"1691556939080"} 2023-08-09 04:55:39,082 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-09 04:55:39,083 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-09 04:55:39,083 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556939083"}]},"ts":"1691556939083"} 2023-08-09 04:55:39,085 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-08-09 04:55:39,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:39,092 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:39,092 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:39,092 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-09 04:55:39,092 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-09 04:55:39,092 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:39,092 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN}] 2023-08-09 04:55:39,094 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN 2023-08-09 04:55:39,095 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40127,1691556935817; forceNewPlan=false, retain=false 2023-08-09 04:55:39,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:39,192 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:55:39,245 INFO [jenkins-hbase4:38949] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-09 04:55:39,246 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:39,247 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556939246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556939246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556939246"}]},"ts":"1691556939246"} 2023-08-09 04:55:39,250 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure baf34b839cf96847099b70ccf7c3d7ea, server=jenkins-hbase4.apache.org,40127,1691556935817}] 2023-08-09 04:55:39,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:39,379 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-09 04:55:39,380 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-09 04:55:39,380 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-09 04:55:39,381 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-09 04:55:39,405 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:39,405 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:39,407 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35496, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:39,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:39,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40127%2C1691556935817.jenkins-hbase4.apache.org%2C40127%2C1691556935817.regiongroup-1, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:39,435 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:39,435 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:39,435 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:39,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817/jenkins-hbase4.apache.org%2C40127%2C1691556935817.jenkins-hbase4.apache.org%2C40127%2C1691556935817.regiongroup-1.1691556939418 2023-08-09 04:55:39,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK]] 2023-08-09 04:55:39,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => baf34b839cf96847099b70ccf7c3d7ea, NAME => 't1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:39,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:39,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,441 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,443 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1 2023-08-09 04:55:39,443 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1 2023-08-09 04:55:39,444 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baf34b839cf96847099b70ccf7c3d7ea columnFamilyName cf1 2023-08-09 04:55:39,444 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(310): Store=baf34b839cf96847099b70ccf7c3d7ea/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:39,444 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,446 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/norep 2023-08-09 04:55:39,446 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/norep 2023-08-09 04:55:39,447 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baf34b839cf96847099b70ccf7c3d7ea columnFamilyName norep 2023-08-09 04:55:39,448 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(310): Store=baf34b839cf96847099b70ccf7c3d7ea/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:39,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:55:39,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:39,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:39,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened baf34b839cf96847099b70ccf7c3d7ea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68488356, jitterRate=0.02055603265762329}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:55:39,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:55:39,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., pid=11, masterSystemTime=1691556939405 2023-08-09 04:55:39,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:39,464 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:39,464 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556939464"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556939464"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556939464"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556939464"}]},"ts":"1691556939464"} 2023-08-09 04:55:39,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-09 04:55:39,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure baf34b839cf96847099b70ccf7c3d7ea, server=jenkins-hbase4.apache.org,40127,1691556935817 in 217 msec 2023-08-09 04:55:39,472 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-09 04:55:39,472 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN in 377 msec 2023-08-09 04:55:39,473 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-09 04:55:39,473 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556939473"}]},"ts":"1691556939473"} 2023-08-09 04:55:39,475 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-08-09 04:55:39,478 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-09 04:55:39,481 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 454 msec 2023-08-09 04:55:39,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-09 04:55:39,637 INFO [Listener at localhost/38001] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-08-09 04:55:39,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:39,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-08-09 04:55:39,642 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-09 04:55:39,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-08-09 04:55:39,643 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-09 04:55:39,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:39,646 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:39,646 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a empty. 2023-08-09 04:55:39,647 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:39,647 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-08-09 04:55:39,667 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-08-09 04:55:39,668 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 629be257c9d8450abcf2763c66f8a37a, NAME => 't2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.tmp 2023-08-09 04:55:39,689 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:39,689 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 629be257c9d8450abcf2763c66f8a37a, disabling compactions & flushes 2023-08-09 04:55:39,689 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:39,689 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:39,689 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. after waiting 0 ms 2023-08-09 04:55:39,689 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:39,689 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:39,689 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:55:39,692 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-09 04:55:39,693 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556939693"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691556939693"}]},"ts":"1691556939693"} 2023-08-09 04:55:39,695 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-09 04:55:39,696 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-09 04:55:39,696 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556939696"}]},"ts":"1691556939696"} 2023-08-09 04:55:39,698 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-08-09 04:55:39,702 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-09 04:55:39,704 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-09 04:55:39,704 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-09 04:55:39,704 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-09 04:55:39,704 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-09 04:55:39,704 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-09 04:55:39,704 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN}] 2023-08-09 04:55:39,706 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN 2023-08-09 04:55:39,707 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40127,1691556935817; forceNewPlan=false, retain=false 2023-08-09 04:55:39,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:39,857 INFO [jenkins-hbase4:38949] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-09 04:55:39,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=629be257c9d8450abcf2763c66f8a37a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:39,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556939859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556939859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556939859"}]},"ts":"1691556939859"} 2023-08-09 04:55:39,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 629be257c9d8450abcf2763c66f8a37a, server=jenkins-hbase4.apache.org,40127,1691556935817}] 2023-08-09 04:55:39,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:40,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:40,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 629be257c9d8450abcf2763c66f8a37a, NAME => 't2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:40,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:40,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,021 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,023 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1 2023-08-09 04:55:40,023 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1 2023-08-09 04:55:40,023 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 629be257c9d8450abcf2763c66f8a37a columnFamilyName cf1 2023-08-09 04:55:40,024 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(310): Store=629be257c9d8450abcf2763c66f8a37a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:40,024 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,026 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/norep 2023-08-09 04:55:40,026 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/norep 2023-08-09 04:55:40,026 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 629be257c9d8450abcf2763c66f8a37a columnFamilyName norep 2023-08-09 04:55:40,027 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(310): Store=629be257c9d8450abcf2763c66f8a37a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:40,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:55:40,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:40,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-09 04:55:40,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 629be257c9d8450abcf2763c66f8a37a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68726432, jitterRate=0.024103641510009766}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:55:40,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:55:40,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a., pid=14, masterSystemTime=1691556940014 2023-08-09 04:55:40,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:40,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:40,038 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=629be257c9d8450abcf2763c66f8a37a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:40,039 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556940038"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556940038"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556940038"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556940038"}]},"ts":"1691556940038"} 2023-08-09 04:55:40,043 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-09 04:55:40,043 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 629be257c9d8450abcf2763c66f8a37a, server=jenkins-hbase4.apache.org,40127,1691556935817 in 179 msec 2023-08-09 04:55:40,046 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-09 04:55:40,046 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN in 339 msec 2023-08-09 04:55:40,047 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-09 04:55:40,047 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691556940047"}]},"ts":"1691556940047"} 2023-08-09 04:55:40,049 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-08-09 04:55:40,052 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-09 04:55:40,055 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 414 msec 2023-08-09 04:55:40,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38949] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-09 04:55:40,249 INFO [Listener at localhost/38001] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-08-09 04:55:40,249 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x6774d1fc to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:40,255 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56dba6b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:40,255 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x7974c63d to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:40,265 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@384e13de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:40,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:63198:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-08-09 04:55:40,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-08-09 04:55:40,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-09 04:55:40,306 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-09 04:55:40,306 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@1f315e18check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:40,311 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@1f315e18check-peer-cluster-id0x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:40,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@1f315e18check-peer-cluster-id-0x101e814d6e10014 connected 2023-08-09 04:55:40,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-09 04:55:40,414 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@1f315e18check-peer-cluster-id-0x101e814d6e10014, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:40,422 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:40,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-08-09 04:55:40,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-09 04:55:40,611 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41117] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-08-09 04:55:40,611 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42251] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-08-09 04:55:40,613 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-08-09 04:55:40,613 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-08-09 04:55:40,626 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-09 04:55:40,627 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-09 04:55:40,646 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:55:40,646 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:55:40,673 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-08-09 04:55:40,673 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-08-09 04:55:40,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] master.HMaster(3641): Remote procedure done, pid=16 2023-08-09 04:55:40,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38065] master.HMaster(3641): Remote procedure done, pid=17 2023-08-09 04:55:40,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38065] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,41117,1691556931790 suceeded 2023-08-09 04:55:40,676 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38065] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,42251,1691556931838 suceeded 2023-08-09 04:55:40,682 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 243 msec 2023-08-09 04:55:40,682 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-09 04:55:40,683 INFO [PEWorker-3] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:63198:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-08-09 04:55:40,683 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 243 msec 2023-08-09 04:55:40,684 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 402 msec 2023-08-09 04:55:40,700 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1b7d430a to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:40,707 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3b2c8493, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:40,708 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x6d3de437 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:40,710 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:40,717 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:40,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e10017 connected 2023-08-09 04:55:40,718 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2356dd7a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:40,719 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:40,722 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:40,726 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-09 04:55:40,726 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:40,726 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0 2023-08-09 04:55:40,726 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-09 04:55:40,726 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:40,727 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1 2023-08-09 04:55:40,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e10018 connected 2023-08-09 04:55:40,731 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:40,731 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:40,732 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1 2023-08-09 04:55:40,732 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0 2023-08-09 04:55:40,734 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0 2023-08-09 04:55:40,734 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1 2023-08-09 04:55:40,734 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:40,734 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:40,735 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1 2023-08-09 04:55:40,736 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0 2023-08-09 04:55:40,810 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:40,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:40,817 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:40,817 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:40,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38065] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-09 04:55:40,899 DEBUG [Listener at localhost/38001] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-08-09 04:55:40,901 DEBUG [hconnection-0x546edf4d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:40,903 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34540, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:40,919 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:40,930 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-09 04:55:40,931 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 487 2023-08-09 04:55:40,931 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-09 04:55:40,931 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 487 2023-08-09 04:55:40,931 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/4=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153, lastWalPosition=487, nbRowKeys=1, nbHFiles=0, heapSize=208, lastSeqIds={}, endOfFile=false,usedBufferSize=208] 2023-08-09 04:55:40,936 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:40,953 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:40,956 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36958, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-08-09 04:55:40,959 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:40,960 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] zookeeper.ReadOnlyZKClient(139): Connect 0x11df337a to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:40,966 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4e299303, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:40,979 DEBUG [hconnection-0x409a2e44-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:40,980 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50206, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,005 DEBUG [hconnection-0x409a2e44-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,007 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51654, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,031 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,042 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-09 04:55:41,045 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,049 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,063 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 7084 2023-08-09 04:55:41,063 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 63 WAL entries eligible for replication 2023-08-09 04:55:41,063 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 7084 2023-08-09 04:55:41,063 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/5=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/6=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/7=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/8=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/9=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/10=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/11=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/12=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/13=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/14=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/15=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/16=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/17=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/18=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/19=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/20=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/21=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/22=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/23=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/24=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/25=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/26=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/27=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/28=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/29=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/30=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/31=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/32=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/33=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/34=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/35=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/36=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/37=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/38=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/39=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/40=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/41=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/42=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/43=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/44=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/45=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/46=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/47=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/48=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/49=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/50=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/51=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/52=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/53=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/54=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/55=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/56=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/57=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/58=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/59=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/60=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/61=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/62=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/63=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/64=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/65=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/66=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/67=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153, lastWalPosition=7084, nbRowKeys=63, nbHFiles=0, heapSize=13104, lastSeqIds={}, endOfFile=false,usedBufferSize=13104] 2023-08-09 04:55:41,071 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:41,082 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,140 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,157 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 2989 2023-08-09 04:55:41,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 25 WAL entries eligible for replication 2023-08-09 04:55:41,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 2989 2023-08-09 04:55:41,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2ccff9a5354735fa815894d8a555d598/4=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/5=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/6=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/7=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/8=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/9=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/10=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/11=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/12=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/13=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/14=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/15=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/16=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/17=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/18=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/19=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/20=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/21=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/22=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/23=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/24=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/25=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/26=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/27=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/28=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=2989, nbRowKeys=25, nbHFiles=0, heapSize=5200, lastSeqIds={}, endOfFile=false,usedBufferSize=5200] 2023-08-09 04:55:41,161 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:41,176 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 36 WAL entries eligible for replication 2023-08-09 04:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/68=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/69=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/70=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/71=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/72=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/73=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/74=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/75=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/76=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/77=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/78=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/79=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/80=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/81=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/82=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/83=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/84=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/85=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/86=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/87=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/88=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/89=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/90=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/91=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/92=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/93=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/94=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/95=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/96=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/97=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/98=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/99=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/100=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/101=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/102=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153, lastWalPosition=10975, nbRowKeys=36, nbHFiles=0, heapSize=7488, lastSeqIds={}, endOfFile=false,usedBufferSize=7488] 2023-08-09 04:55:41,185 DEBUG [SinkThread-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:41,187 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50210, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-08-09 04:55:41,188 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46479] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:41,190 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46479] zookeeper.ReadOnlyZKClient(139): Connect 0x13732452 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:41,201 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46479] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@134783f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:41,206 DEBUG [hconnection-0x222a3316-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,207 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50220, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,214 DEBUG [hconnection-0x222a3316-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,216 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51670, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,223 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46479] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,251 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,266 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,278 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8344 2023-08-09 04:55:41,278 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 51 WAL entries eligible for replication 2023-08-09 04:55:41,278 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8344 2023-08-09 04:55:41,279 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2ccff9a5354735fa815894d8a555d598/29=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/30=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/31=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/32=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/33=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/34=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/35=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/36=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/37=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/38=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/39=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/40=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/41=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/42=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/43=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/44=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/45=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/46=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/47=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/48=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/49=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/50=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/51=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/52=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/53=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/54=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/55=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/56=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/57=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/58=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/59=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/60=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/61=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/62=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/63=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/64=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/65=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/66=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/67=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/68=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/69=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/70=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/71=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/72=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/73=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/74=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/75=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/76=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/77=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/78=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/79=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=8344, nbRowKeys=51, nbHFiles=0, heapSize=10608, lastSeqIds={}, endOfFile=false,usedBufferSize=10608] 2023-08-09 04:55:41,280 DEBUG [SinkThread-4] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:41,281 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51672, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-08-09 04:55:41,283 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40127] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:41,284 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40127] zookeeper.ReadOnlyZKClient(139): Connect 0x06085ea9 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:41,295 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40127] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@8cfb018, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:41,301 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,302 DEBUG [hconnection-0xcf035d9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,303 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50230, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,311 DEBUG [hconnection-0xcf035d9-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,312 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,317 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40127] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,360 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,376 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,397 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15700 2023-08-09 04:55:41,397 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 69 WAL entries eligible for replication 2023-08-09 04:55:41,397 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15700 2023-08-09 04:55:41,397 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2ccff9a5354735fa815894d8a555d598/80=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/81=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/82=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/83=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/84=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/85=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/86=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/87=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/88=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/89=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/90=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/91=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/92=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/93=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/94=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/95=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/96=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/97=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/98=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/99=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/100=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/101=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/102=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/103=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/104=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/105=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/106=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/107=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/108=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/109=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/110=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/111=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/112=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/113=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/114=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/115=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/116=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/117=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/118=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/119=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/120=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/121=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/122=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/123=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/124=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/125=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/126=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/127=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/128=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/129=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/130=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/131=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/132=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/133=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/134=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/135=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/136=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/137=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/138=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/139=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/140=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/141=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/142=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/143=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/144=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/145=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/146=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/147=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/148=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=15700, nbRowKeys=69, nbHFiles=0, heapSize=14352, lastSeqIds={}, endOfFile=false,usedBufferSize=14352] 2023-08-09 04:55:41,403 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:41,411 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,415 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42521] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,445 INFO [jenkins-hbase4:33953Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:41,452 INFO [jenkins-hbase4:46479Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:41,452 INFO [jenkins-hbase4:46479Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 36 2023-08-09 04:55:41,458 INFO [jenkins-hbase4:40127Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:41,458 INFO [jenkins-hbase4:40127Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 51 2023-08-09 04:55:41,458 INFO [jenkins-hbase4:42521Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:41,458 INFO [jenkins-hbase4:42521Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 158 2023-08-09 04:55:41,468 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,483 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,509 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:41,509 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 55 WAL entries eligible for replication 2023-08-09 04:55:41,509 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:41,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2ccff9a5354735fa815894d8a555d598/149=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/150=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/151=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/152=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/153=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/154=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/155=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/156=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/157=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/158=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/159=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/160=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/161=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/162=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/163=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/164=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/165=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/166=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/167=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/168=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/169=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/170=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/171=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/172=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/173=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/174=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/175=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/176=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/177=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/178=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/179=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/180=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/181=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/182=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/183=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/184=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/185=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/186=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/187=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/188=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/189=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/190=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/191=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/192=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/193=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/194=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/195=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/196=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/197=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/198=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/199=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/200=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/201=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/202=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=21752, nbRowKeys=55, nbHFiles=0, heapSize=11440, lastSeqIds={}, endOfFile=false,usedBufferSize=11440] 2023-08-09 04:55:41,513 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46479] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:41,518 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,523 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46479] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:41,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,591 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,619 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:41,626 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,682 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,697 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,726 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:41,732 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,788 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,802 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,833 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:41,846 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,895 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:41,908 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:41,938 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:41,952 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:41,972 DEBUG [hconnection-0x5018485e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,973 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50234, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:41,978 DEBUG [hconnection-0x5018485e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:41,980 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51696, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:42,001 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,003 DEBUG [Listener at localhost/38001] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-08-09 04:55:42,003 INFO [Listener at localhost/38001] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-09 04:55:42,003 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x037ad110 to 127.0.0.1:63198 2023-08-09 04:55:42,004 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,004 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-09 04:55:42,005 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(257): Found active master hash=130007629, stopped=false 2023-08-09 04:55:42,005 INFO [Listener at localhost/38001] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:42,008 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:55:42,008 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:55:42,008 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:55:42,008 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:55:42,008 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:55:42,008 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:42,009 INFO [Listener at localhost/38001] procedure2.ProcedureExecutor(629): Stopping 2023-08-09 04:55:42,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:42,009 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2cbb9e7e to 127.0.0.1:63198 2023-08-09 04:55:42,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:42,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:42,009 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:42,012 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40127,1691556935817' ***** 2023-08-09 04:55:42,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:42,012 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1064): Closing user regions 2023-08-09 04:55:42,012 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:55:42,013 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33953,1691556935846' ***** 2023-08-09 04:55:42,013 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:55:42,013 INFO [RS:0;jenkins-hbase4:40127] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:55:42,013 INFO [RS:1;jenkins-hbase4:33953] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:55:42,013 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46479,1691556935882' ***** 2023-08-09 04:55:42,013 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:55:42,013 INFO [RS:1;jenkins-hbase4:33953] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:55:42,014 INFO [RS:0;jenkins-hbase4:40127] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:55:42,014 INFO [RS:1;jenkins-hbase4:33953] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:55:42,014 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:42,014 DEBUG [RS:1;jenkins-hbase4:33953] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c1428f5 to 127.0.0.1:63198 2023-08-09 04:55:42,014 DEBUG [RS:1;jenkins-hbase4:33953] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,014 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33953,1691556935846; all regions closed. 2023-08-09 04:55:42,013 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:55:42,013 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:55:42,014 INFO [RS:2;jenkins-hbase4:46479] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:55:42,014 INFO [RS:0;jenkins-hbase4:40127] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:55:42,020 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(3305): Received CLOSE for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:42,020 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:55:42,014 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42521,1691556935911' ***** 2023-08-09 04:55:42,022 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:55:42,021 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(3305): Received CLOSE for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:42,019 INFO [RS:2;jenkins-hbase4:46479] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:55:42,023 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:42,024 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57b7b3e7 to 127.0.0.1:63198 2023-08-09 04:55:42,024 DEBUG [RS:0;jenkins-hbase4:40127] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,024 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-09 04:55:42,024 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1478): Online Regions={baf34b839cf96847099b70ccf7c3d7ea=t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., 629be257c9d8450abcf2763c66f8a37a=t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.} 2023-08-09 04:55:42,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing baf34b839cf96847099b70ccf7c3d7ea, disabling compactions & flushes 2023-08-09 04:55:42,024 INFO [RS:3;jenkins-hbase4:42521] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:55:42,024 INFO [RS:3;jenkins-hbase4:42521] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:55:42,024 INFO [RS:3;jenkins-hbase4:42521] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:55:42,024 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(3305): Received CLOSE for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:42,024 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,024 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:55:42,024 INFO [RS:2;jenkins-hbase4:46479] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:55:42,024 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,025 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:42,025 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7bb110ac to 127.0.0.1:63198 2023-08-09 04:55:42,025 DEBUG [RS:2;jenkins-hbase4:46479] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,025 INFO [RS:2;jenkins-hbase4:46479] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:55:42,025 INFO [RS:2;jenkins-hbase4:46479] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:55:42,025 INFO [RS:2;jenkins-hbase4:46479] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:55:42,025 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-09 04:55:42,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:42,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:42,026 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1504): Waiting on 629be257c9d8450abcf2763c66f8a37a, baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:42,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. after waiting 0 ms 2023-08-09 04:55:42,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:42,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing baf34b839cf96847099b70ccf7c3d7ea 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-08-09 04:55:42,027 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,029 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:42,029 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7a502592 to 127.0.0.1:63198 2023-08-09 04:55:42,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36954931527a871a4ac33b21c04863f3, disabling compactions & flushes 2023-08-09 04:55:42,031 DEBUG [RS:3;jenkins-hbase4:42521] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:42,031 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-09 04:55:42,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:42,031 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1478): Online Regions={36954931527a871a4ac33b21c04863f3=hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.} 2023-08-09 04:55:42,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. after waiting 0 ms 2023-08-09 04:55:42,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:42,031 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1504): Waiting on 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:42,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 36954931527a871a4ac33b21c04863f3 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-09 04:55:42,034 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,036 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,036 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-09 04:55:42,038 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-08-09 04:55:42,038 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:55:42,038 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-09 04:55:42,038 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:55:42,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:55:42,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:55:42,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:55:42,039 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-08-09 04:55:42,047 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,051 DEBUG [RS:1;jenkins-hbase4:33953] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,051 INFO [RS:1;jenkins-hbase4:33953] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33953%2C1691556935846.jenkins-hbase4.apache.org%2C33953%2C1691556935846.regiongroup-0:(num 1691556936554) 2023-08-09 04:55:42,051 DEBUG [RS:1;jenkins-hbase4:33953] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,051 INFO [RS:1;jenkins-hbase4:33953] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,052 INFO [RS:1;jenkins-hbase4:33953] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-09 04:55:42,052 INFO [RS:1;jenkins-hbase4:33953] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:55:42,052 INFO [RS:1;jenkins-hbase4:33953] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:55:42,052 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:42,052 INFO [RS:1;jenkins-hbase4:33953] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:55:42,054 INFO [RS:1;jenkins-hbase4:33953] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33953 2023-08-09 04:55:42,058 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,115 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/.tmp/info/33bb63d4ef714b58996b9beecc59a90a 2023-08-09 04:55:42,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/.tmp/cf1/98fba2c9188748208109db5c299fb897 2023-08-09 04:55:42,129 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/info/e298eec93fb24463bb4dc7984a19b330 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,130 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10017, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,130 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33953,1691556935846 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,130 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,132 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33953,1691556935846] 2023-08-09 04:55:42,132 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33953,1691556935846; numProcessing=1 2023-08-09 04:55:42,133 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33953,1691556935846 already deleted, retry=false 2023-08-09 04:55:42,134 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33953,1691556935846 expired; onlineServers=3 2023-08-09 04:55:42,134 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,152 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,168 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/.tmp/info/33bb63d4ef714b58996b9beecc59a90a as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info/33bb63d4ef714b58996b9beecc59a90a 2023-08-09 04:55:42,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/.tmp/cf1/98fba2c9188748208109db5c299fb897 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/98fba2c9188748208109db5c299fb897 2023-08-09 04:55:42,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info/33bb63d4ef714b58996b9beecc59a90a, entries=2, sequenceid=6, filesize=4.8 K 2023-08-09 04:55:42,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 36954931527a871a4ac33b21c04863f3 in 172ms, sequenceid=6, compaction requested=false 2023-08-09 04:55:42,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/98fba2c9188748208109db5c299fb897, entries=100, sequenceid=7, filesize=8.3 K 2023-08-09 04:55:42,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for baf34b839cf96847099b70ccf7c3d7ea in 183ms, sequenceid=7, compaction requested=false 2023-08-09 04:55:42,226 DEBUG [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1504): Waiting on 629be257c9d8450abcf2763c66f8a37a, baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:42,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-09 04:55:42,231 DEBUG [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1504): Waiting on 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:42,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-08-09 04:55:42,237 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:42,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:42,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:55:42,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:55:42,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:42,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:42,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 629be257c9d8450abcf2763c66f8a37a, disabling compactions & flushes 2023-08-09 04:55:42,239 DEBUG [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-09 04:55:42,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:42,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:42,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. after waiting 0 ms 2023-08-09 04:55:42,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:42,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 629be257c9d8450abcf2763c66f8a37a 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-08-09 04:55:42,246 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/table/f077c49e23464cb69defed46ad23be28 2023-08-09 04:55:42,248 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,257 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/info/e298eec93fb24463bb4dc7984a19b330 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/e298eec93fb24463bb4dc7984a19b330 2023-08-09 04:55:42,260 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,273 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/e298eec93fb24463bb4dc7984a19b330, entries=30, sequenceid=19, filesize=8.1 K 2023-08-09 04:55:42,274 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/table/f077c49e23464cb69defed46ad23be28 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table/f077c49e23464cb69defed46ad23be28 2023-08-09 04:55:42,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/.tmp/cf1/1b482211d41c43d88278a7605ec6ae58 2023-08-09 04:55:42,277 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,282 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table/f077c49e23464cb69defed46ad23be28, entries=6, sequenceid=19, filesize=4.8 K 2023-08-09 04:55:42,283 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 244ms, sequenceid=19, compaction requested=false 2023-08-09 04:55:42,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/.tmp/cf1/1b482211d41c43d88278a7605ec6ae58 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/1b482211d41c43d88278a7605ec6ae58 2023-08-09 04:55:42,293 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-08-09 04:55:42,294 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-09 04:55:42,294 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:55:42,295 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:55:42,295 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-09 04:55:42,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/1b482211d41c43d88278a7605ec6ae58, entries=200, sequenceid=8, filesize=12.2 K 2023-08-09 04:55:42,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 629be257c9d8450abcf2763c66f8a37a in 57ms, sequenceid=8, compaction requested=false 2023-08-09 04:55:42,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-08-09 04:55:42,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:42,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:55:42,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:42,343 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,352 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,364 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,382 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,412 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-09 04:55:42,412 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-09 04:55:42,413 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:42,413 INFO [RS:1;jenkins-hbase4:33953] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33953,1691556935846; zookeeper connection closed. 2023-08-09 04:55:42,413 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:33953-0x101e814d6e10009, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:42,415 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@55418981] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@55418981 2023-08-09 04:55:42,419 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-09 04:55:42,420 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-09 04:55:42,421 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-09 04:55:42,421 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-09 04:55:42,426 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40127,1691556935817; all regions closed. 2023-08-09 04:55:42,432 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42521,1691556935911; all regions closed. 2023-08-09 04:55:42,437 DEBUG [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,437 INFO [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40127%2C1691556935817.jenkins-hbase4.apache.org%2C40127%2C1691556935817.regiongroup-0:(num 1691556936570) 2023-08-09 04:55:42,441 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46479,1691556935882; all regions closed. 2023-08-09 04:55:42,444 DEBUG [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,444 INFO [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42521%2C1691556935911.jenkins-hbase4.apache.org%2C42521%2C1691556935911.regiongroup-0:(num 1691556936566) 2023-08-09 04:55:42,448 DEBUG [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,449 INFO [RS:0;jenkins-hbase4:40127] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40127%2C1691556935817.jenkins-hbase4.apache.org%2C40127%2C1691556935817.regiongroup-1:(num 1691556939418) 2023-08-09 04:55:42,449 DEBUG [RS:0;jenkins-hbase4:40127] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,449 INFO [RS:0;jenkins-hbase4:40127] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,451 INFO [RS:0;jenkins-hbase4:40127] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-09 04:55:42,452 INFO [RS:0;jenkins-hbase4:40127] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:55:42,453 INFO [RS:0;jenkins-hbase4:40127] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:55:42,453 INFO [RS:0;jenkins-hbase4:40127] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:55:42,452 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:42,457 DEBUG [RS:0;jenkins-hbase4:40127] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06085ea9 to 127.0.0.1:63198 2023-08-09 04:55:42,457 DEBUG [RS:0;jenkins-hbase4:40127] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,457 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,458 INFO [RS:0;jenkins-hbase4:40127] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40127 2023-08-09 04:55:42,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:42,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:42,472 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10017, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,472 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:55:42,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40127,1691556935817 2023-08-09 04:55:42,473 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40127,1691556935817] 2023-08-09 04:55:42,473 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40127,1691556935817; numProcessing=2 2023-08-09 04:55:42,475 DEBUG [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,475 INFO [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46479%2C1691556935882.meta:.meta(num 1691556936794) 2023-08-09 04:55:42,476 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40127,1691556935817 already deleted, retry=false 2023-08-09 04:55:42,476 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40127,1691556935817 expired; onlineServers=2 2023-08-09 04:55:42,476 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,477 DEBUG [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,477 INFO [RS:3;jenkins-hbase4:42521] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42521%2C1691556935911.jenkins-hbase4.apache.org%2C42521%2C1691556935911.regiongroup-1:(num 1691556937503) 2023-08-09 04:55:42,477 DEBUG [RS:3;jenkins-hbase4:42521] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,477 INFO [RS:3;jenkins-hbase4:42521] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,478 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,479 INFO [RS:3;jenkins-hbase4:42521] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-09 04:55:42,479 INFO [RS:3;jenkins-hbase4:42521] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:55:42,479 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:42,479 INFO [RS:3;jenkins-hbase4:42521] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:55:42,479 INFO [RS:3;jenkins-hbase4:42521] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:55:42,481 DEBUG [RS:3;jenkins-hbase4:42521] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11df337a to 127.0.0.1:63198 2023-08-09 04:55:42,481 DEBUG [RS:3;jenkins-hbase4:42521] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,482 INFO [RS:3;jenkins-hbase4:42521] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42521 2023-08-09 04:55:42,488 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:42,488 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,488 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10017, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:42,488 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,42521,1691556935911 2023-08-09 04:55:42,488 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:55:42,492 DEBUG [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:55:42,492 INFO [RS:2;jenkins-hbase4:46479] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46479%2C1691556935882.jenkins-hbase4.apache.org%2C46479%2C1691556935882.regiongroup-0:(num 1691556936560) 2023-08-09 04:55:42,492 DEBUG [RS:2;jenkins-hbase4:46479] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,492 INFO [RS:2;jenkins-hbase4:46479] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:42,492 INFO [RS:2;jenkins-hbase4:46479] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-09 04:55:42,492 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,492 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:42,493 DEBUG [RS:2;jenkins-hbase4:46479] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13732452 to 127.0.0.1:63198 2023-08-09 04:55:42,493 DEBUG [RS:2;jenkins-hbase4:46479] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,494 INFO [RS:2;jenkins-hbase4:46479] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46479 2023-08-09 04:55:42,564 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,581 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,582 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,591 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:42,591 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46479,1691556935882 2023-08-09 04:55:42,592 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42521,1691556935911] 2023-08-09 04:55:42,592 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42521,1691556935911; numProcessing=3 2023-08-09 04:55:42,594 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,42521,1691556935911 already deleted, retry=false 2023-08-09 04:55:42,594 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42521,1691556935911 expired; onlineServers=1 2023-08-09 04:55:42,594 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46479,1691556935882] 2023-08-09 04:55:42,594 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46479,1691556935882; numProcessing=4 2023-08-09 04:55:42,596 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46479,1691556935882 already deleted, retry=false 2023-08-09 04:55:42,596 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46479,1691556935882 expired; onlineServers=0 2023-08-09 04:55:42,596 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38949,1691556935777' ***** 2023-08-09 04:55:42,596 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-09 04:55:42,597 DEBUG [M:0;jenkins-hbase4:38949] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3805662, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:42,597 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38949,1691556935777 2023-08-09 04:55:42,597 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38949,1691556935777; all regions closed. 2023-08-09 04:55:42,597 DEBUG [M:0;jenkins-hbase4:38949] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:42,597 DEBUG [M:0;jenkins-hbase4:38949] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-09 04:55:42,597 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-09 04:55:42,598 DEBUG [M:0;jenkins-hbase4:38949] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-09 04:55:42,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556936205] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556936205,5,FailOnTimeoutGroup] 2023-08-09 04:55:42,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556936207] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556936207,5,FailOnTimeoutGroup] 2023-08-09 04:55:42,599 INFO [M:0;jenkins-hbase4:38949] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-09 04:55:42,600 INFO [M:0;jenkins-hbase4:38949] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-09 04:55:42,600 INFO [M:0;jenkins-hbase4:38949] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-09 04:55:42,600 DEBUG [M:0;jenkins-hbase4:38949] master.HMaster(1512): Stopping service threads 2023-08-09 04:55:42,600 INFO [M:0;jenkins-hbase4:38949] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-09 04:55:42,601 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-09 04:55:42,602 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:42,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:42,602 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,604 ERROR [M:0;jenkins-hbase4:38949] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-08-09 04:55:42,605 INFO [M:0;jenkins-hbase4:38949] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-09 04:55:42,605 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-09 04:55:42,605 DEBUG [M:0;jenkins-hbase4:38949] zookeeper.ZKUtil(398): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-09 04:55:42,605 WARN [M:0;jenkins-hbase4:38949] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-09 04:55:42,605 INFO [M:0;jenkins-hbase4:38949] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-09 04:55:42,606 INFO [M:0;jenkins-hbase4:38949] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-09 04:55:42,607 DEBUG [M:0;jenkins-hbase4:38949] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:55:42,607 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:42,607 DEBUG [M:0;jenkins-hbase4:38949] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:42,607 DEBUG [M:0;jenkins-hbase4:38949] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:55:42,607 DEBUG [M:0;jenkins-hbase4:38949] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:42,607 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.39 KB heapSize=65.18 KB 2023-08-09 04:55:42,624 INFO [M:0;jenkins-hbase4:38949] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.39 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/242b42096ae04f3190e328926c0ee389 2023-08-09 04:55:42,630 DEBUG [M:0;jenkins-hbase4:38949] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/242b42096ae04f3190e328926c0ee389 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/242b42096ae04f3190e328926c0ee389 2023-08-09 04:55:42,635 INFO [M:0;jenkins-hbase4:38949] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/242b42096ae04f3190e328926c0ee389, entries=14, sequenceid=116, filesize=8.1 K 2023-08-09 04:55:42,636 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegion(2948): Finished flush of dataSize ~55.39 KB/56720, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=116, compaction requested=false 2023-08-09 04:55:42,638 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:42,639 DEBUG [M:0;jenkins-hbase4:38949] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:42,642 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:42,642 INFO [M:0;jenkins-hbase4:38949] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-09 04:55:42,643 INFO [M:0;jenkins-hbase4:38949] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38949 2023-08-09 04:55:42,645 DEBUG [M:0;jenkins-hbase4:38949] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,38949,1691556935777 already deleted, retry=false 2023-08-09 04:55:42,669 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,686 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,687 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,707 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,714 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:42,714 INFO [RS:2;jenkins-hbase4:46479] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46479,1691556935882; zookeeper connection closed. 2023-08-09 04:55:42,715 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:46479-0x101e814d6e1000a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:42,716 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3f783466] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3f783466 2023-08-09 04:55:42,774 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,790 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,791 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,812 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,879 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:42,895 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:42,895 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:42,915 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:42,915 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42521-0x101e814d6e1000b, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:42,915 INFO [RS:3;jenkins-hbase4:42521] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42521,1691556935911; zookeeper connection closed. 2023-08-09 04:55:42,916 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@29e8bb5b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@29e8bb5b 2023-08-09 04:55:42,919 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:42,984 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:43,000 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:43,000 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:43,015 INFO [RS:0;jenkins-hbase4:40127] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40127,1691556935817; zookeeper connection closed. 2023-08-09 04:55:43,015 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,015 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:40127-0x101e814d6e10008, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,016 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2f5679f5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2f5679f5 2023-08-09 04:55:43,017 INFO [Listener at localhost/38001] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-08-09 04:55:43,023 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:43,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:43,104 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:43,104 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:43,128 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-09 04:55:43,193 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:43,209 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-09 04:55:43,209 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-09 04:55:43,215 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,215 INFO [M:0;jenkins-hbase4:38949] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38949,1691556935777; zookeeper connection closed. 2023-08-09 04:55:43,215 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38949-0x101e814d6e10007, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,237 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-08-09 04:55:43,237 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-09 04:55:43,237 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-08-09 04:55:43,237 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153, lastWalPosition=14321, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-08-09 04:55:43,239 INFO [SinkThread-7] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-08-09 04:55:43,240 WARN [SinkThread-7] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:55:43,240 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-09 04:55:43,246 INFO [Listener at localhost/38001] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-09 04:55:43,246 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x512eb72d to 127.0.0.1:63198 2023-08-09 04:55:43,247 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,247 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-09 04:55:43,247 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(257): Found active master hash=1385005374, stopped=false 2023-08-09 04:55:43,247 INFO [Listener at localhost/38001] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:43,249 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-09 04:55:43,249 INFO [Listener at localhost/38001] procedure2.ProcedureExecutor(629): Stopping 2023-08-09 04:55:43,249 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:43,249 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22200881 to 127.0.0.1:63198 2023-08-09 04:55:43,251 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,251 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41117,1691556931790' ***** 2023-08-09 04:55:43,251 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:55:43,251 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42251,1691556931838' ***** 2023-08-09 04:55:43,251 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:55:43,249 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-09 04:55:43,251 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:55:43,252 INFO [RS:1;jenkins-hbase4:42251] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:55:43,252 INFO [RS:1;jenkins-hbase4:42251] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:55:43,252 INFO [RS:1;jenkins-hbase4:42251] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:55:43,252 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(3305): Received CLOSE for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:55:43,252 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:55:43,252 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-09 04:55:43,252 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:55:43,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:55:43,256 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(3305): Received CLOSE for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:43,256 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:43,256 DEBUG [RS:1;jenkins-hbase4:42251] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04bb76aa to 127.0.0.1:63198 2023-08-09 04:55:43,256 DEBUG [RS:1;jenkins-hbase4:42251] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,257 INFO [RS:1;jenkins-hbase4:42251] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:55:43,257 INFO [RS:1;jenkins-hbase4:42251] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:55:43,257 INFO [RS:1;jenkins-hbase4:42251] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:55:43,257 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-09 04:55:43,257 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-09 04:55:43,257 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 2ccff9a5354735fa815894d8a555d598=t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598., 4d1b412f72764eba378319d319bc7848=t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.} 2023-08-09 04:55:43,258 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:55:43,258 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1504): Waiting on 1588230740, 2ccff9a5354735fa815894d8a555d598, 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:43,258 INFO [RS:0;jenkins-hbase4:41117] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:55:43,258 INFO [RS:0;jenkins-hbase4:41117] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:55:43,259 INFO [RS:0;jenkins-hbase4:41117] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:55:43,259 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(3305): Received CLOSE for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:43,258 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:55:43,259 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:55:43,259 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:55:43,259 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:55:43,259 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:55:43,259 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-08-09 04:55:43,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2ccff9a5354735fa815894d8a555d598, disabling compactions & flushes 2023-08-09 04:55:43,262 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:43,262 INFO [jenkins-hbase4:42251Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=4120B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0]: currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385 at position: 21752 walGroup [jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1]: currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153 at position: 10975 2023-08-09 04:55:43,262 DEBUG [RS:0;jenkins-hbase4:41117] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7129712f to 127.0.0.1:63198 2023-08-09 04:55:43,262 INFO [jenkins-hbase4:41117Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1]: currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 at position: 600 walGroup [jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0]: currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385 at position: -1 2023-08-09 04:55:43,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:43,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cb1d40dc2c341948bac0897d809dc4a0, disabling compactions & flushes 2023-08-09 04:55:43,262 DEBUG [RS:0;jenkins-hbase4:41117] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:43,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:43,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:43,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. after waiting 0 ms 2023-08-09 04:55:43,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:43,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2ccff9a5354735fa815894d8a555d598 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-08-09 04:55:43,263 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-09 04:55:43,264 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1478): Online Regions={cb1d40dc2c341948bac0897d809dc4a0=hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.} 2023-08-09 04:55:43,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. after waiting 0 ms 2023-08-09 04:55:43,264 DEBUG [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1504): Waiting on cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:55:43,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:43,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing cb1d40dc2c341948bac0897d809dc4a0 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-09 04:55:43,293 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-09 04:55:43,294 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-09 04:55:43,317 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/info/ef3f847fb4f644f68aaa332e672d35f4 2023-08-09 04:55:43,318 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:43,319 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-08-09 04:55:43,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/.tmp/info/e2b6b9e74b5f418ca048abb7dca929af 2023-08-09 04:55:43,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-08-09 04:55:43,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346, lastWalPosition=845, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:43,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/cf1/7798f4cf09d8411fbc9c9641fbb654f8 2023-08-09 04:55:43,322 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-09 04:55:43,322 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-09 04:55:43,322 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-09 04:55:43,322 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2ccff9a5354735fa815894d8a555d598/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=28643, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-08-09 04:55:43,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/.tmp/info/e2b6b9e74b5f418ca048abb7dca929af as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info/e2b6b9e74b5f418ca048abb7dca929af 2023-08-09 04:55:43,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7798f4cf09d8411fbc9c9641fbb654f8 2023-08-09 04:55:43,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info/e2b6b9e74b5f418ca048abb7dca929af, entries=2, sequenceid=6, filesize=4.8 K 2023-08-09 04:55:43,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for cb1d40dc2c341948bac0897d809dc4a0 in 75ms, sequenceid=6, compaction requested=false 2023-08-09 04:55:43,341 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:43,341 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:43,346 INFO [SinkThread-8] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-08-09 04:55:43,356 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-08-09 04:55:43,368 WARN [SinkThread-8] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:55:43,368 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-09 04:55:43,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/rep_barrier/42fdd228bc2447059dcd2931fd60c30e 2023-08-09 04:55:43,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-09 04:55:43,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/norep/eed3659271f644c2a30750a1a1bf1f78 2023-08-09 04:55:43,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:43,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cb1d40dc2c341948bac0897d809dc4a0: 2023-08-09 04:55:43,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:55:43,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eed3659271f644c2a30750a1a1bf1f78 2023-08-09 04:55:43,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/cf1/7798f4cf09d8411fbc9c9641fbb654f8 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1/7798f4cf09d8411fbc9c9641fbb654f8 2023-08-09 04:55:43,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7798f4cf09d8411fbc9c9641fbb654f8 2023-08-09 04:55:43,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1/7798f4cf09d8411fbc9c9641fbb654f8, entries=200, sequenceid=206, filesize=11.9 K 2023-08-09 04:55:43,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/norep/eed3659271f644c2a30750a1a1bf1f78 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep/eed3659271f644c2a30750a1a1bf1f78 2023-08-09 04:55:43,422 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/table/3a83a2e674af4ac38a084dae47afdf21 2023-08-09 04:55:43,424 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:43,426 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-09 04:55:43,426 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,426 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-09 04:55:43,429 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-09 04:55:43,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eed3659271f644c2a30750a1a1bf1f78 2023-08-09 04:55:43,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep/eed3659271f644c2a30750a1a1bf1f78, entries=101, sequenceid=206, filesize=8.3 K 2023-08-09 04:55:43,434 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/info/ef3f847fb4f644f68aaa332e672d35f4 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info/ef3f847fb4f644f68aaa332e672d35f4 2023-08-09 04:55:43,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 2ccff9a5354735fa815894d8a555d598 in 171ms, sequenceid=206, compaction requested=false 2023-08-09 04:55:43,444 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info/ef3f847fb4f644f68aaa332e672d35f4, entries=30, sequenceid=19, filesize=8.1 K 2023-08-09 04:55:43,445 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/rep_barrier/42fdd228bc2447059dcd2931fd60c30e as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier/42fdd228bc2447059dcd2931fd60c30e 2023-08-09 04:55:43,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-08-09 04:55:43,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:43,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2ccff9a5354735fa815894d8a555d598: 2023-08-09 04:55:43,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:55:43,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4d1b412f72764eba378319d319bc7848, disabling compactions & flushes 2023-08-09 04:55:43,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:43,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:43,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. after waiting 0 ms 2023-08-09 04:55:43,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:43,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4d1b412f72764eba378319d319bc7848 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-08-09 04:55:43,453 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier/42fdd228bc2447059dcd2931fd60c30e, entries=2, sequenceid=19, filesize=4.9 K 2023-08-09 04:55:43,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/table/3a83a2e674af4ac38a084dae47afdf21 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table/3a83a2e674af4ac38a084dae47afdf21 2023-08-09 04:55:43,458 DEBUG [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1504): Waiting on 1588230740, 4d1b412f72764eba378319d319bc7848 2023-08-09 04:55:43,460 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table/3a83a2e674af4ac38a084dae47afdf21, entries=6, sequenceid=19, filesize=4.8 K 2023-08-09 04:55:43,461 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 202ms, sequenceid=19, compaction requested=false 2023-08-09 04:55:43,461 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-08-09 04:55:43,462 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,462 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-08-09 04:55:43,465 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41117,1691556931790; all regions closed. 2023-08-09 04:55:43,485 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 not finished, retry = 0 2023-08-09 04:55:43,499 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-08-09 04:55:43,500 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-09 04:55:43,501 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:55:43,501 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:55:43,501 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-09 04:55:43,502 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/cf1/0860b4b531b648f5bdf2c07ea6e9cf3a 2023-08-09 04:55:43,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0860b4b531b648f5bdf2c07ea6e9cf3a 2023-08-09 04:55:43,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/norep/e77efce3e5ac4e678334d3d7285a6da5 2023-08-09 04:55:43,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e77efce3e5ac4e678334d3d7285a6da5 2023-08-09 04:55:43,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/cf1/0860b4b531b648f5bdf2c07ea6e9cf3a as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1/0860b4b531b648f5bdf2c07ea6e9cf3a 2023-08-09 04:55:43,530 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:55:43,534 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-08-09 04:55:43,534 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,534 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-08-09 04:55:43,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0860b4b531b648f5bdf2c07ea6e9cf3a 2023-08-09 04:55:43,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1/0860b4b531b648f5bdf2c07ea6e9cf3a, entries=100, sequenceid=106, filesize=8.2 K 2023-08-09 04:55:43,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/norep/e77efce3e5ac4e678334d3d7285a6da5 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep/e77efce3e5ac4e678334d3d7285a6da5 2023-08-09 04:55:43,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e77efce3e5ac4e678334d3d7285a6da5 2023-08-09 04:55:43,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep/e77efce3e5ac4e678334d3d7285a6da5, entries=51, sequenceid=106, filesize=6.5 K 2023-08-09 04:55:43,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for 4d1b412f72764eba378319d319bc7848 in 95ms, sequenceid=106, compaction requested=false 2023-08-09 04:55:43,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-08-09 04:55:43,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:43,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4d1b412f72764eba378319d319bc7848: 2023-08-09 04:55:43,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:55:43,566 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-08-09 04:55:43,566 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,575 INFO [SinkThread-9] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-08-09 04:55:43,575 WARN [SinkThread-9] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:55:43,575 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-09 04:55:43,588 DEBUG [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:55:43,588 INFO [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1:(num 1691556934346) 2023-08-09 04:55:43,593 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385 not finished, retry = 0 2023-08-09 04:55:43,639 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-08-09 04:55:43,659 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42251,1691556931838; all regions closed. 2023-08-09 04:55:43,665 DEBUG [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:55:43,665 INFO [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42251%2C1691556931838.meta:.meta(num 1691556933573) 2023-08-09 04:55:43,670 DEBUG [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:55:43,670 INFO [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0:(num 1691556933385) 2023-08-09 04:55:43,675 DEBUG [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:55:43,675 INFO [RS:1;jenkins-hbase4:42251] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1:(num 1691556938153) 2023-08-09 04:55:43,675 DEBUG [RS:1;jenkins-hbase4:42251] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,675 INFO [RS:1;jenkins-hbase4:42251] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:43,675 INFO [RS:1;jenkins-hbase4:42251] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-09 04:55:43,675 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:43,675 INFO [RS:1;jenkins-hbase4:42251] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-09 04:55:43,695 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346, it was closed cleanly currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 at position: 1430 2023-08-09 04:55:43,695 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 2023-08-09 04:55:43,695 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,696 DEBUG [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:55:43,696 INFO [RS:0;jenkins-hbase4:41117] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0:(num 1691556933385) 2023-08-09 04:55:43,696 DEBUG [RS:0;jenkins-hbase4:41117] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,696 INFO [RS:0;jenkins-hbase4:41117] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:55:43,697 INFO [RS:0;jenkins-hbase4:41117] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-09 04:55:43,697 INFO [RS:0;jenkins-hbase4:41117] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:55:43,697 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:43,697 INFO [RS:0;jenkins-hbase4:41117] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:55:43,697 INFO [RS:0;jenkins-hbase4:41117] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:55:43,697 INFO [RS:0;jenkins-hbase4:41117] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-09 04:55:43,699 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385, it was closed cleanly currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385 at position: 0 2023-08-09 04:55:43,699 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385 2023-08-09 04:55:43,699 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:43,699 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385, lastWalPosition=-1, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-08-09 04:55:43,700 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0.1691556933385] 2023-08-09 04:55:43,741 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385 was moved to hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385 2023-08-09 04:55:43,778 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10017, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,778 DEBUG [RS:1;jenkins-hbase4:42251] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b7d430a to 127.0.0.1:63198 2023-08-09 04:55:43,778 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10017, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,778 DEBUG [RS:1;jenkins-hbase4:42251] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,878 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10018, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:43,879 DEBUG [RS:0;jenkins-hbase4:41117] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d3de437 to 127.0.0.1:63198 2023-08-09 04:55:43,879 DEBUG [RS:0;jenkins-hbase4:41117] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,881 INFO [RS:1;jenkins-hbase4:42251] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1 terminated 2023-08-09 04:55:43,881 INFO [RS:1;jenkins-hbase4:42251] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1 terminated 2023-08-09 04:55:43,883 INFO [RS:1;jenkins-hbase4:42251] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42251 2023-08-09 04:55:43,885 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:43,885 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:43,885 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:55:43,885 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:55:43,885 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:55:43,886 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:43,886 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42251,1691556931838; numProcessing=1 2023-08-09 04:55:43,887 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42251,1691556931838 already deleted, retry=false 2023-08-09 04:55:43,887 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42251,1691556931838 expired; onlineServers=1 2023-08-09 04:55:43,979 INFO [RS:0;jenkins-hbase4:41117] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1 terminated 2023-08-09 04:55:43,980 INFO [RS:0;jenkins-hbase4:41117] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-0,1 terminated 2023-08-09 04:55:43,981 INFO [RS:0;jenkins-hbase4:41117] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41117 2023-08-09 04:55:43,984 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:43,984 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:55:43,985 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41117,1691556931790] 2023-08-09 04:55:43,985 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41117,1691556931790; numProcessing=2 2023-08-09 04:55:43,986 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,41117,1691556931790 already deleted, retry=false 2023-08-09 04:55:43,986 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41117,1691556931790 expired; onlineServers=0 2023-08-09 04:55:43,986 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38065,1691556930543' ***** 2023-08-09 04:55:43,986 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-09 04:55:43,987 DEBUG [M:0;jenkins-hbase4:38065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@13b08b05, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:43,987 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38065,1691556930543 2023-08-09 04:55:43,987 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38065,1691556930543; all regions closed. 2023-08-09 04:55:43,987 DEBUG [M:0;jenkins-hbase4:38065] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:43,987 DEBUG [M:0;jenkins-hbase4:38065] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-09 04:55:43,987 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-09 04:55:43,988 DEBUG [M:0;jenkins-hbase4:38065] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-09 04:55:43,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556933060] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556933060,5,FailOnTimeoutGroup] 2023-08-09 04:55:43,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556933061] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556933061,5,FailOnTimeoutGroup] 2023-08-09 04:55:43,988 INFO [M:0;jenkins-hbase4:38065] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-09 04:55:43,990 INFO [M:0;jenkins-hbase4:38065] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-09 04:55:43,990 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-08-09 04:55:43,990 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:55:43,990 INFO [M:0;jenkins-hbase4:38065] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-09 04:55:43,990 DEBUG [M:0;jenkins-hbase4:38065] master.HMaster(1512): Stopping service threads 2023-08-09 04:55:43,990 INFO [M:0;jenkins-hbase4:38065] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-09 04:55:43,991 ERROR [M:0;jenkins-hbase4:38065] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-08-09 04:55:43,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:55:43,991 INFO [M:0;jenkins-hbase4:38065] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-09 04:55:43,991 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-09 04:55:43,991 DEBUG [M:0;jenkins-hbase4:38065] zookeeper.ZKUtil(398): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-08-09 04:55:43,991 WARN [M:0;jenkins-hbase4:38065] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-09 04:55:43,991 INFO [M:0;jenkins-hbase4:38065] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-09 04:55:43,992 INFO [M:0;jenkins-hbase4:38065] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-09 04:55:43,992 DEBUG [M:0;jenkins-hbase4:38065] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:55:43,992 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:43,992 DEBUG [M:0;jenkins-hbase4:38065] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:43,993 DEBUG [M:0;jenkins-hbase4:38065] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:55:43,993 DEBUG [M:0;jenkins-hbase4:38065] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:43,993 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.02 KB heapSize=72.60 KB 2023-08-09 04:55:44,009 INFO [M:0;jenkins-hbase4:38065] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.02 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/cf9c8330e63f4d86a56e6426f27a9048 2023-08-09 04:55:44,016 DEBUG [M:0;jenkins-hbase4:38065] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/cf9c8330e63f4d86a56e6426f27a9048 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/cf9c8330e63f4d86a56e6426f27a9048 2023-08-09 04:55:44,021 INFO [M:0;jenkins-hbase4:38065] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/cf9c8330e63f4d86a56e6426f27a9048, entries=17, sequenceid=136, filesize=8.5 K 2023-08-09 04:55:44,022 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegion(2948): Finished flush of dataSize ~61.02 KB/62484, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=136, compaction requested=false 2023-08-09 04:55:44,024 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:55:44,024 DEBUG [M:0;jenkins-hbase4:38065] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:44,027 INFO [M:0;jenkins-hbase4:38065] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-09 04:55:44,027 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:55:44,028 INFO [M:0;jenkins-hbase4:38065] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38065 2023-08-09 04:55:44,033 DEBUG [M:0;jenkins-hbase4:38065] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,38065,1691556930543 already deleted, retry=false 2023-08-09 04:55:44,079 INFO [RS:1;jenkins-hbase4:42251] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42251,1691556931838; zookeeper connection closed. 2023-08-09 04:55:44,079 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:44,079 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:42251-0x101e814d6e10002, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:44,079 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6080e86c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6080e86c 2023-08-09 04:55:44,279 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:44,279 INFO [M:0;jenkins-hbase4:38065] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38065,1691556930543; zookeeper connection closed. 2023-08-09 04:55:44,280 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): master:38065-0x101e814d6e10000, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:44,304 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:55:44,380 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:44,380 DEBUG [Listener at localhost/38459-EventThread] zookeeper.ZKWatcher(600): regionserver:41117-0x101e814d6e10001, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:55:44,380 INFO [RS:0;jenkins-hbase4:41117] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41117,1691556931790; zookeeper connection closed. 2023-08-09 04:55:44,383 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@39d6bcf4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@39d6bcf4 2023-08-09 04:55:44,383 INFO [Listener at localhost/38001] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-08-09 04:55:44,385 INFO [Listener at localhost/38001] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:44,385 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:44,385 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:44,385 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:44,385 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:44,385 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:44,386 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:44,391 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42053 2023-08-09 04:55:44,392 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:44,393 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:44,394 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=master:42053 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:44,399 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:420530x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:44,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:42053-0x101e814d6e1001c connected 2023-08-09 04:55:44,406 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:44,407 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:44,407 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:44,408 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42053 2023-08-09 04:55:44,410 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42053 2023-08-09 04:55:44,412 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42053 2023-08-09 04:55:44,416 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42053 2023-08-09 04:55:44,416 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42053 2023-08-09 04:55:44,417 INFO [Listener at localhost/38001] master.HMaster(444): hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027, hbase.cluster.distributed=false 2023-08-09 04:55:44,418 DEBUG [pool-456-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-08-09 04:55:44,436 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:55:44,436 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:44,436 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:44,437 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:55:44,437 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:55:44,437 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:55:44,437 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:55:44,439 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45045 2023-08-09 04:55:44,439 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:55:44,442 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:55:44,443 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:44,445 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:44,446 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45045 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:44,452 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:450450x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:44,454 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:450450x0, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:55:44,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45045-0x101e814d6e1001d connected 2023-08-09 04:55:44,455 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:55:44,456 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:55:44,460 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45045 2023-08-09 04:55:44,460 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45045 2023-08-09 04:55:44,464 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45045 2023-08-09 04:55:44,467 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45045 2023-08-09 04:55:44,467 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45045 2023-08-09 04:55:44,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,470 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-09 04:55:44,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:44,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:55:44,472 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:44,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-09 04:55:44,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,42053,1691556944384 from backup master directory 2023-08-09 04:55:44,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-09 04:55:44,476 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,476 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-09 04:55:44,476 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:44,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:55:44,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5df54625 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:44,534 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@431dd417, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:44,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:55:44,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-09 04:55:44,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:44,536 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:44,544 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777 to hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777-dead as it is dead 2023-08-09 04:55:44,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777-dead/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 2023-08-09 04:55:44,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777-dead/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 after 5ms 2023-08-09 04:55:44,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777-dead/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 to hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 2023-08-09 04:55:44,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38949,1691556935777-dead 2023-08-09 04:55:44,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:44,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/oldWALs, maxLogs=10 2023-08-09 04:55:44,576 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:44,576 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:44,576 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:44,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 2023-08-09 04:55:44,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK]] 2023-08-09 04:55:44,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:44,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:44,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:44,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:44,583 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:44,584 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-09 04:55:44,584 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-09 04:55:44,591 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/242b42096ae04f3190e328926c0ee389 2023-08-09 04:55:44,592 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:44,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-09 04:55:44,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 2023-08-09 04:55:44,601 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 2023-08-09 04:55:44,602 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38949%2C1691556935777.jenkins-hbase4.apache.org%2C38949%2C1691556935777.regiongroup-0.1691556936088 2023-08-09 04:55:44,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:55:44,611 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-08-09 04:55:44,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74159442, jitterRate=0.10506179928779602}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:55:44,612 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:55:44,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-09 04:55:44,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-09 04:55:44,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-09 04:55:44,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-09 04:55:44,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-09 04:55:44,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-09 04:55:44,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-09 04:55:44,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-09 04:55:44,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-09 04:55:44,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-09 04:55:44,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-09 04:55:44,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 19 msec 2023-08-09 04:55:44,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-09 04:55:44,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-09 04:55:44,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46479,1691556935882, table=hbase:meta, region=1588230740 2023-08-09 04:55:44,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-08-09 04:55:44,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40127,1691556935817 already deleted, retry=false 2023-08-09 04:55:44,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,40127,1691556935817 on jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,40127,1691556935817, splitWal=true, meta=false 2023-08-09 04:55:44,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,40127,1691556935817 (carryingMeta=false) jenkins-hbase4.apache.org,40127,1691556935817/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@295a97e6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:55:44,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,42521,1691556935911 already deleted, retry=false 2023-08-09 04:55:44,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42521,1691556935911 on jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42521,1691556935911, splitWal=true, meta=false 2023-08-09 04:55:44,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,42521,1691556935911 (carryingMeta=false) jenkins-hbase4.apache.org,42521,1691556935911/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@5d1db0c4[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:55:44,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33953,1691556935846 already deleted, retry=false 2023-08-09 04:55:44,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33953,1691556935846 on jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33953,1691556935846, splitWal=true, meta=false 2023-08-09 04:55:44,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,33953,1691556935846 (carryingMeta=false) jenkins-hbase4.apache.org,33953,1691556935846/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@2085e926[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:55:44,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46479,1691556935882 already deleted, retry=false 2023-08-09 04:55:44,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46479,1691556935882 on jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46479,1691556935882, splitWal=true, meta=true 2023-08-09 04:55:44,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,46479,1691556935882 (carryingMeta=true) jenkins-hbase4.apache.org,46479,1691556935882/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@26b942c1[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:55:44,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-09 04:55:44,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-09 04:55:44,684 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-09 04:55:44,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-09 04:55:44,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-09 04:55:44,686 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-09 04:55:44,686 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-09 04:55:44,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-09 04:55:44,694 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:44,694 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:55:44,694 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:55:44,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,42053,1691556944384, sessionid=0x101e814d6e1001c, setting cluster-up flag (Was=false) 2023-08-09 04:55:44,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-09 04:55:44,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-09 04:55:44,703 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:55:44,703 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.hbase-snapshot/.tmp 2023-08-09 04:55:44,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:44,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:44,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:44,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:55:44,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-09 04:55:44,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:44,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691556974705 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-09 04:55:44,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-09 04:55:44,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-09 04:55:44,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-09 04:55:44,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-09 04:55:44,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-09 04:55:44,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556944711,5,FailOnTimeoutGroup] 2023-08-09 04:55:44,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556944711,5,FailOnTimeoutGroup] 2023-08-09 04:55:44,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-09 04:55:44,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691556944712, completionTime=-1 2023-08-09 04:55:44,713 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-09 04:55:44,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-09 04:55:44,714 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42521,1691556935911; numProcessing=1 2023-08-09 04:55:44,714 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33953,1691556935846; numProcessing=2 2023-08-09 04:55:44,714 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40127,1691556935817; numProcessing=3 2023-08-09 04:55:44,714 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46479,1691556935882; numProcessing=4 2023-08-09 04:55:44,714 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40127,1691556935817, splitWal=true, meta=false 2023-08-09 04:55:44,714 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42521,1691556935911, splitWal=true, meta=false 2023-08-09 04:55:44,714 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33953,1691556935846, splitWal=true, meta=false 2023-08-09 04:55:44,714 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46479,1691556935882, splitWal=true, meta=true 2023-08-09 04:55:44,716 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46479,1691556935882, splitWal=true, meta=true, isMeta: true 2023-08-09 04:55:44,718 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882-splitting 2023-08-09 04:55:44,719 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882-splitting dir is empty, no logs to split. 2023-08-09 04:55:44,719 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46479,1691556935882 WAL count=0, meta=true 2023-08-09 04:55:44,723 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882-splitting dir is empty, no logs to split. 2023-08-09 04:55:44,723 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46479,1691556935882 WAL count=0, meta=true 2023-08-09 04:55:44,723 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46479,1691556935882 WAL splitting is done? wals=0, meta=true 2023-08-09 04:55:44,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-09 04:55:44,725 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-09 04:55:44,727 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:55:44,768 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(951): ClusterId : 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:44,769 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:55:44,770 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:55:44,771 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:55:44,773 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:55:44,775 DEBUG [RS:0;jenkins-hbase4:45045] zookeeper.ReadOnlyZKClient(139): Connect 0x25e5c999 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:44,779 DEBUG [RS:0;jenkins-hbase4:45045] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b222355, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:44,779 DEBUG [RS:0;jenkins-hbase4:45045] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f1da2e9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:55:44,788 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45045 2023-08-09 04:55:44,788 INFO [RS:0;jenkins-hbase4:45045] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:55:44,788 INFO [RS:0;jenkins-hbase4:45045] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:55:44,788 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:55:44,789 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,42053,1691556944384 with isa=jenkins-hbase4.apache.org/172.31.14.131:45045, startcode=1691556944435 2023-08-09 04:55:44,789 DEBUG [RS:0;jenkins-hbase4:45045] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:55:44,790 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41751, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:55:44,791 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=42053] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:44,792 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:55:44,792 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39381 2023-08-09 04:55:44,792 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:55:44,793 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:55:44,794 DEBUG [RS:0;jenkins-hbase4:45045] zookeeper.ZKUtil(162): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:44,794 WARN [RS:0;jenkins-hbase4:45045] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:55:44,794 INFO [RS:0;jenkins-hbase4:45045] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:44,794 INFO [RS:0;jenkins-hbase4:45045] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:44,794 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:44,795 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45045,1691556944435] 2023-08-09 04:55:44,798 DEBUG [RS:0;jenkins-hbase4:45045] zookeeper.ZKUtil(162): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:44,799 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:44,799 INFO [RS:0;jenkins-hbase4:45045] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:55:44,800 INFO [RS:0;jenkins-hbase4:45045] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:55:44,801 INFO [RS:0;jenkins-hbase4:45045] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:55:44,801 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,801 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:55:44,802 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,803 DEBUG [RS:0;jenkins-hbase4:45045] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:55:44,805 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,805 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,805 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-09 04:55:44,817 INFO [RS:0;jenkins-hbase4:45045] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:55:44,817 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45045,1691556944435-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:44,829 INFO [RS:0;jenkins-hbase4:45045] regionserver.Replication(203): jenkins-hbase4.apache.org,45045,1691556944435 started 2023-08-09 04:55:44,829 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45045,1691556944435, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45045, sessionid=0x101e814d6e1001d 2023-08-09 04:55:44,829 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:55:44,829 DEBUG [RS:0;jenkins-hbase4:45045] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:44,829 DEBUG [RS:0;jenkins-hbase4:45045] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45045,1691556944435' 2023-08-09 04:55:44,829 DEBUG [RS:0;jenkins-hbase4:45045] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-09 04:55:44,830 DEBUG [RS:0;jenkins-hbase4:45045] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-09 04:55:44,830 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:55:44,830 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:55:44,830 DEBUG [RS:0;jenkins-hbase4:45045] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:44,830 DEBUG [RS:0;jenkins-hbase4:45045] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45045,1691556944435' 2023-08-09 04:55:44,830 DEBUG [RS:0;jenkins-hbase4:45045] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-09 04:55:44,831 DEBUG [RS:0;jenkins-hbase4:45045] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-09 04:55:44,831 DEBUG [RS:0;jenkins-hbase4:45045] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:55:44,831 INFO [RS:0;jenkins-hbase4:45045] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:55:44,831 INFO [RS:0;jenkins-hbase4:45045] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:55:44,877 DEBUG [jenkins-hbase4:42053] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-09 04:55:44,878 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45045,1691556944435, state=OPENING 2023-08-09 04:55:44,880 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-09 04:55:44,880 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-09 04:55:44,880 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45045,1691556944435}] 2023-08-09 04:55:44,931 INFO [RS:0;jenkins-hbase4:45045] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:44,933 INFO [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45045%2C1691556944435.jenkins-hbase4.apache.org%2C45045%2C1691556944435.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:44,950 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:44,950 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:44,950 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:44,952 INFO [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435/jenkins-hbase4.apache.org%2C45045%2C1691556944435.jenkins-hbase4.apache.org%2C45045%2C1691556944435.regiongroup-0.1691556944934 2023-08-09 04:55:44,952 DEBUG [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK]] 2023-08-09 04:55:45,034 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:45,034 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:45,036 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39086, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:45,039 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-09 04:55:45,039 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:45,040 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:45,040 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:45,041 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45045%2C1691556944435.meta, suffix=.meta, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:45,058 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:55:45,058 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:45,058 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:45,060 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435/jenkins-hbase4.apache.org%2C45045%2C1691556944435.meta.1691556945042.meta 2023-08-09 04:55:45,062 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK]] 2023-08-09 04:55:45,063 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:45,063 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-09 04:55:45,063 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-09 04:55:45,064 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-09 04:55:45,064 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-09 04:55:45,064 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:45,064 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-09 04:55:45,064 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-09 04:55:45,066 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:55:45,067 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:55:45,067 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:55:45,067 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:55:45,074 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/e298eec93fb24463bb4dc7984a19b330 2023-08-09 04:55:45,074 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:45,074 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:55:45,075 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:45,075 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:55:45,076 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:55:45,076 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:45,076 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:55:45,077 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:55:45,077 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:55:45,077 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:55:45,083 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table/f077c49e23464cb69defed46ad23be28 2023-08-09 04:55:45,083 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:45,084 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:55:45,085 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:55:45,088 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:55:45,089 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:55:45,090 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73810446, jitterRate=0.09986135363578796}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:55:45,090 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:55:45,091 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1691556945034 2023-08-09 04:55:45,095 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-09 04:55:45,096 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-09 04:55:45,096 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45045,1691556944435, state=OPEN 2023-08-09 04:55:45,098 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-09 04:55:45,099 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-09 04:55:45,101 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-08-09 04:55:45,101 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45045,1691556944435 in 219 msec 2023-08-09 04:55:45,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-08-09 04:55:45,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 377 msec 2023-08-09 04:55:46,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-08-09 04:55:47,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-08-09 04:55:49,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-09 04:55:49,221 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-09 04:55:49,225 DEBUG [hconnection-0x129e3cf8-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:49,226 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39092, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:49,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=36954931527a871a4ac33b21c04863f3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42521,1691556935911, regionLocation=jenkins-hbase4.apache.org,42521,1691556935911, openSeqNum=2 2023-08-09 04:55:49,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40127,1691556935817, regionLocation=jenkins-hbase4.apache.org,40127,1691556935817, openSeqNum=2 2023-08-09 04:55:49,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=629be257c9d8450abcf2763c66f8a37a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40127,1691556935817, regionLocation=jenkins-hbase4.apache.org,40127,1691556935817, openSeqNum=2 2023-08-09 04:55:49,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-09 04:55:49,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691557009236 2023-08-09 04:55:49,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691557069236 2023-08-09 04:55:49,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-08-09 04:55:49,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42053,1691556944384-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:49,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42053,1691556944384-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:49,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42053,1691556944384-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:49,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:42053, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:49,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-09 04:55:49,261 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33953,1691556935846 had 0 regions 2023-08-09 04:55:49,261 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,40127,1691556935817 had 2 regions 2023-08-09 04:55:49,261 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46479,1691556935882 had 1 regions 2023-08-09 04:55:49,261 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42521,1691556935911 had 1 regions 2023-08-09 04:55:49,261 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. is NOT online; state={36954931527a871a4ac33b21c04863f3 state=OPEN, ts=1691556949235, server=jenkins-hbase4.apache.org,42521,1691556935911}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-09 04:55:49,262 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33953,1691556935846, splitWal=true, meta=false, isMeta: false 2023-08-09 04:55:49,263 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46479,1691556935882, splitWal=true, meta=true, isMeta: false 2023-08-09 04:55:49,263 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40127,1691556935817, splitWal=true, meta=false, isMeta: false 2023-08-09 04:55:49,263 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42521,1691556935911, splitWal=true, meta=false, isMeta: false 2023-08-09 04:55:49,264 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846-splitting 2023-08-09 04:55:49,266 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,266 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,33953,1691556935846 WAL count=0, meta=false 2023-08-09 04:55:49,267 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,267 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,46479,1691556935882 WAL count=0, meta=false 2023-08-09 04:55:49,267 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817-splitting 2023-08-09 04:55:49,268 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,42521,1691556935911/hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., unknown_server=jenkins-hbase4.apache.org,40127,1691556935817/t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., unknown_server=jenkins-hbase4.apache.org,40127,1691556935817/t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:49,268 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,268 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,40127,1691556935817 WAL count=0, meta=false 2023-08-09 04:55:49,268 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911-splitting 2023-08-09 04:55:49,269 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,269 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,42521,1691556935911 WAL count=0, meta=false 2023-08-09 04:55:49,270 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,270 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,33953,1691556935846 WAL count=0, meta=false 2023-08-09 04:55:49,270 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33953,1691556935846 WAL splitting is done? wals=0, meta=false 2023-08-09 04:55:49,271 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,46479,1691556935882-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,271 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,46479,1691556935882 WAL count=0, meta=false 2023-08-09 04:55:49,271 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46479,1691556935882 WAL splitting is done? wals=0, meta=false 2023-08-09 04:55:49,272 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,272 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,40127,1691556935817 WAL count=0, meta=false 2023-08-09 04:55:49,272 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,40127,1691556935817 WAL splitting is done? wals=0, meta=false 2023-08-09 04:55:49,273 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,33953,1691556935846 failed, ignore...File hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,33953,1691556935846-splitting does not exist. 2023-08-09 04:55:49,273 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911-splitting dir is empty, no logs to split. 2023-08-09 04:55:49,273 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,42521,1691556935911 WAL count=0, meta=false 2023-08-09 04:55:49,273 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42521,1691556935911 WAL splitting is done? wals=0, meta=false 2023-08-09 04:55:49,274 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46479,1691556935882 after splitting done 2023-08-09 04:55:49,274 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46479,1691556935882 from processing; numProcessing=3 2023-08-09 04:55:49,275 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,40127,1691556935817 failed, ignore...File hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,40127,1691556935817-splitting does not exist. 2023-08-09 04:55:49,275 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33953,1691556935846 after splitting done 2023-08-09 04:55:49,275 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33953,1691556935846 from processing; numProcessing=2 2023-08-09 04:55:49,276 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,42521,1691556935911 failed, ignore...File hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,42521,1691556935911-splitting does not exist. 2023-08-09 04:55:49,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN}, {pid=22, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN}] 2023-08-09 04:55:49,276 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46479,1691556935882, splitWal=true, meta=true in 4.6070 sec 2023-08-09 04:55:49,277 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN}] 2023-08-09 04:55:49,277 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33953,1691556935846, splitWal=true, meta=false in 4.6100 sec 2023-08-09 04:55:49,277 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN 2023-08-09 04:55:49,277 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN 2023-08-09 04:55:49,278 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN 2023-08-09 04:55:49,278 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:55:49,278 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:55:49,279 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:55:49,279 DEBUG [jenkins-hbase4:42053] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-09 04:55:49,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=629be257c9d8450abcf2763c66f8a37a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:49,281 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:49,282 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556949281"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556949281"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556949281"}]},"ts":"1691556949281"} 2023-08-09 04:55:49,282 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556949281"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556949281"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556949281"}]},"ts":"1691556949281"} 2023-08-09 04:55:49,282 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=36954931527a871a4ac33b21c04863f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:49,282 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556949282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556949282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556949282"}]},"ts":"1691556949282"} 2023-08-09 04:55:49,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure 629be257c9d8450abcf2763c66f8a37a, server=jenkins-hbase4.apache.org,45045,1691556944435}] 2023-08-09 04:55:49,288 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure baf34b839cf96847099b70ccf7c3d7ea, server=jenkins-hbase4.apache.org,45045,1691556944435}] 2023-08-09 04:55:49,289 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 36954931527a871a4ac33b21c04863f3, server=jenkins-hbase4.apache.org,45045,1691556944435}] 2023-08-09 04:55:49,385 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:55:49,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:49,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:55:49,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45045%2C1691556944435.jenkins-hbase4.apache.org%2C45045%2C1691556944435.regiongroup-1, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:55:49,446 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:49,446 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36954931527a871a4ac33b21c04863f3, NAME => 'hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:49,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:49,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,448 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,449 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info 2023-08-09 04:55:49,449 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info 2023-08-09 04:55:49,450 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36954931527a871a4ac33b21c04863f3 columnFamilyName info 2023-08-09 04:55:49,460 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:55:49,460 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info/33bb63d4ef714b58996b9beecc59a90a 2023-08-09 04:55:49,460 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:55:49,460 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:55:49,460 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(310): Store=36954931527a871a4ac33b21c04863f3/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:49,461 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435/jenkins-hbase4.apache.org%2C45045%2C1691556944435.jenkins-hbase4.apache.org%2C45045%2C1691556944435.regiongroup-1.1691556949446 2023-08-09 04:55:49,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK]] 2023-08-09 04:55:49,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => baf34b839cf96847099b70ccf7c3d7ea, NAME => 't1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:49,465 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:49,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,466 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,467 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1 2023-08-09 04:55:49,467 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1 2023-08-09 04:55:49,468 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:55:49,468 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baf34b839cf96847099b70ccf7c3d7ea columnFamilyName cf1 2023-08-09 04:55:49,469 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36954931527a871a4ac33b21c04863f3; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71760281, jitterRate=0.06931151449680328}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:55:49,469 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:55:49,470 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., pid=26, masterSystemTime=1691556949439 2023-08-09 04:55:49,472 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:49,472 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:55:49,473 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=36954931527a871a4ac33b21c04863f3, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:49,473 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556949472"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556949472"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556949472"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556949472"}]},"ts":"1691556949472"} 2023-08-09 04:55:49,475 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/98fba2c9188748208109db5c299fb897 2023-08-09 04:55:49,476 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(310): Store=baf34b839cf96847099b70ccf7c3d7ea/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:49,476 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-08-09 04:55:49,477 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/norep 2023-08-09 04:55:49,477 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/norep 2023-08-09 04:55:49,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 36954931527a871a4ac33b21c04863f3, server=jenkins-hbase4.apache.org,45045,1691556944435 in 185 msec 2023-08-09 04:55:49,477 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baf34b839cf96847099b70ccf7c3d7ea columnFamilyName norep 2023-08-09 04:55:49,478 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(310): Store=baf34b839cf96847099b70ccf7c3d7ea/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:49,479 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=16 2023-08-09 04:55:49,479 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42521,1691556935911 after splitting done 2023-08-09 04:55:49,479 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN in 200 msec 2023-08-09 04:55:49,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,479 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42521,1691556935911 from processing; numProcessing=1 2023-08-09 04:55:49,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42521,1691556935911, splitWal=true, meta=false in 4.8190 sec 2023-08-09 04:55:49,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:55:49,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:55:49,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened baf34b839cf96847099b70ccf7c3d7ea; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59705150, jitterRate=-0.1103239357471466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:55:49,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:55:49,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., pid=25, masterSystemTime=1691556949439 2023-08-09 04:55:49,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:49,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:55:49,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:49,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 629be257c9d8450abcf2763c66f8a37a, NAME => 't2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:55:49,487 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:49,488 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556949487"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556949487"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556949487"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556949487"}]},"ts":"1691556949487"} 2023-08-09 04:55:49,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:55:49,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,489 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,490 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1 2023-08-09 04:55:49,490 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1 2023-08-09 04:55:49,491 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-08-09 04:55:49,491 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure baf34b839cf96847099b70ccf7c3d7ea, server=jenkins-hbase4.apache.org,45045,1691556944435 in 201 msec 2023-08-09 04:55:49,491 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 629be257c9d8450abcf2763c66f8a37a columnFamilyName cf1 2023-08-09 04:55:49,492 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN in 215 msec 2023-08-09 04:55:49,497 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/1b482211d41c43d88278a7605ec6ae58 2023-08-09 04:55:49,497 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(310): Store=629be257c9d8450abcf2763c66f8a37a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:49,497 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,498 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/norep 2023-08-09 04:55:49,498 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/norep 2023-08-09 04:55:49,498 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 629be257c9d8450abcf2763c66f8a37a columnFamilyName norep 2023-08-09 04:55:49,499 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(310): Store=629be257c9d8450abcf2763c66f8a37a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:55:49,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:55:49,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:55:49,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 629be257c9d8450abcf2763c66f8a37a; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65192221, jitterRate=-0.028560206294059753}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:55:49,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:55:49,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a., pid=24, masterSystemTime=1691556949439 2023-08-09 04:55:49,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:49,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:55:49,507 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=629be257c9d8450abcf2763c66f8a37a, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:55:49,507 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556949506"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556949506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556949506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556949506"}]},"ts":"1691556949506"} 2023-08-09 04:55:49,510 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-08-09 04:55:49,510 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure 629be257c9d8450abcf2763c66f8a37a, server=jenkins-hbase4.apache.org,45045,1691556944435 in 222 msec 2023-08-09 04:55:49,512 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=15 2023-08-09 04:55:49,512 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,40127,1691556935817 after splitting done 2023-08-09 04:55:49,512 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN in 234 msec 2023-08-09 04:55:49,512 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,40127,1691556935817 from processing; numProcessing=0 2023-08-09 04:55:49,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,40127,1691556935817, splitWal=true, meta=false in 4.8600 sec 2023-08-09 04:55:49,829 INFO [jenkins-hbase4:45045Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:50,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-08-09 04:55:50,275 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-09 04:55:50,278 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-09 04:55:50,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.802sec 2023-08-09 04:55:50,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-09 04:55:50,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-09 04:55:50,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-09 04:55:50,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42053,1691556944384-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-09 04:55:50,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42053,1691556944384-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-09 04:55:50,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-09 04:55:50,372 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x6dc8d824 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:50,378 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e2f64f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:50,380 DEBUG [hconnection-0x7022a50-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:50,381 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36916, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:50,384 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-09 04:55:50,384 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6dc8d824 to 127.0.0.1:63198 2023-08-09 04:55:50,384 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:55:50,386 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:42053 after: jenkins-hbase4.apache.org:42053 2023-08-09 04:55:50,386 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x210e00dc to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:50,390 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3748ee07, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:50,799 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-09 04:55:50,800 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-09 04:55:50,801 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-09 04:55:50,802 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-09 04:55:50,893 DEBUG [hconnection-0x571250c9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:50,894 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36922, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:55:50,923 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1691556950923 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:50,930 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication16915569509230x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:50,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1691556950923-0x101e814d6e10022 connected Start Replication Server start 2023-08-09 04:55:50,935 INFO [Listener at localhost/38001] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:55:50,935 INFO [Listener at localhost/38001] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:55:50,947 INFO [Listener at localhost/38001] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-09 04:55:50,948 DEBUG [Listener at localhost/38001] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:55:50,959 INFO [Listener at localhost/38001] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:55:50,960 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,42251,1691556931838, jenkins-hbase4.apache.org,41117,1691556931790] other RSs: [] 2023-08-09 04:55:50,979 DEBUG [Listener at localhost/38001.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x6e505b45 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:50,985 DEBUG [Listener at localhost/38001.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@24d7d14b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:50,985 INFO [Listener at localhost/38001.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:50,988 DEBUG [Listener at localhost/38001.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:50,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e10024 connected 2023-08-09 04:55:50,989 INFO [Listener at localhost/38001.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:53,464 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42251,1691556931838/1's WALs to 1691556950935.syncuptool.replication.org,1234,1 2023-08-09 04:55:53,466 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153 with data PBUF\x08\xDFU 2023-08-09 04:55:53,466 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385 with data PBUF\x08\xF8\xA9\x01 2023-08-09 04:55:53,468 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,42251,1691556931838/1's WALs to 1691556950935.syncuptool.replication.org,1234,1 2023-08-09 04:55:54,829 INFO [jenkins-hbase4:45045Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:54,950 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:55:55,474 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:55,474 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:55,489 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:55,489 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,42251,1691556931838, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:55:55,489 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:55,510 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] zookeeper.ReadOnlyZKClient(139): Connect 0x4bf545ff to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:55,516 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@143c278b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:55,516 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:55,520 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:55,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e10026 connected 2023-08-09 04:55:55,521 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,42251,1691556931838 (queues=2) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:55,521 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0 2023-08-09 04:55:55,525 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42251,1691556931838, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:55,525 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1 2023-08-09 04:55:55,525 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0 2023-08-09 04:55:55,526 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42251,1691556931838, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:55,527 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1 2023-08-09 04:55:55,533 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385 2023-08-09 04:55:55,533 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-09 04:55:55,534 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-09 04:55:55,535 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42251,1691556931838 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2ccff9a5354735fa815894d8a555d598/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-08-09 04:55:55,537 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153 2023-08-09 04:55:55,537 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-09 04:55:55,537 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-09 04:55:55,537 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42251,1691556931838 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-08-09 04:55:55,538 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:55:55,543 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36938, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:55:55,544 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=45045] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:55,544 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=45045] zookeeper.ReadOnlyZKClient(139): Connect 0x6011c76c to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:55,545 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45045] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:55:55,550 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=45045] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6ed6251e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:55,553 DEBUG [hconnection-0x6ae9ac1c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:55:55,554 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36952, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-08-09 04:55:55,568 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45045] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:55,569 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:55,569 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=45045] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:55:55,570 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42251,1691556931838] 2023-08-09 04:55:55,572 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0.1691556933385] 2023-08-09 04:55:55,572 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1.1691556938153] 2023-08-09 04:55:55,573 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42251,1691556931838 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:55,573 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:55,574 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42251,1691556931838 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:55,574 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:55,674 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:55,677 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556950923-0x101e814d6e10022, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691556950935.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:55,678 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-0,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42251,1691556931838 with the following stats: Total replicated edits: 2, current progress: 2023-08-09 04:55:55,778 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42251,1691556931838 2023-08-09 04:55:55,778 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] zookeeper.ZKUtil(243): syncupReplication1691556950923-0x101e814d6e10022, quorum=127.0.0.1:63198, baseZNode=/1 Unable to list children of znode /1/replication/rs/1691556950935.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42251,1691556931838 because node does not exist (not an error) 2023-08-09 04:55:55,779 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838.replicationSource.shipperjenkins-hbase4.apache.org%2C42251%2C1691556931838.jenkins-hbase4.apache.org%2C42251%2C1691556931838.regiongroup-1,1-jenkins-hbase4.apache.org,42251,1691556931838] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42251,1691556931838 with the following stats: Total replicated edits: 2, current progress: 2023-08-09 04:55:57,524 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,41117,1691556931790/1's WALs to 1691556950935.syncuptool.replication.org,1234,1 2023-08-09 04:55:57,526 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 with data PBUF\x08\xCD\x06 2023-08-09 04:55:57,528 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,41117,1691556931790/1's WALs to 1691556950935.syncuptool.replication.org,1234,1 2023-08-09 04:55:59,530 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41117,1691556931790] 2023-08-09 04:55:59,530 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41117,1691556931790] 2023-08-09 04:55:59,541 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41117,1691556931790] 2023-08-09 04:55:59,541 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,41117,1691556931790, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:55:59,541 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:59,567 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790] zookeeper.ReadOnlyZKClient(139): Connect 0x13641a94 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:55:59,573 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1b751d70, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:55:59,573 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:55:59,576 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:55:59,576 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,41117,1691556931790 (queues=1) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:55:59,576 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1 2023-08-09 04:55:59,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e10029 connected 2023-08-09 04:55:59,579 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,41117,1691556931790, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:55:59,588 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1 2023-08-09 04:55:59,602 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346 2023-08-09 04:55:59,602 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:55:59,602 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-09 04:55:59,602 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41117,1691556931790 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-08-09 04:55:59,603 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41117,1691556931790] 2023-08-09 04:55:59,606 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1.1691556934346] 2023-08-09 04:55:59,607 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41117,1691556931790 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:55:59,607 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:59,707 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:59,710 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556950923-0x101e814d6e10022, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691556950935.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41117,1691556931790 2023-08-09 04:55:59,710 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790.replicationSource.shipperjenkins-hbase4.apache.org%2C41117%2C1691556931790.jenkins-hbase4.apache.org%2C41117%2C1691556931790.regiongroup-1,1-jenkins-hbase4.apache.org,41117,1691556931790] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,41117,1691556931790 with the following stats: Total replicated edits: 0, current progress: 2023-08-09 04:55:59,829 INFO [jenkins-hbase4:45045Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:55:59,829 INFO [jenkins-hbase4:45045Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-09 04:56:00,961 INFO [Listener at localhost/38001] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-09 04:56:01,065 DEBUG [Listener at localhost/38001.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10024, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:01,065 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6e505b45 to 127.0.0.1:63198 2023-08-09 04:56:01,066 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:01,177 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:56:01,268 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556950923-0x101e814d6e10022, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:01,268 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556950923-0x101e814d6e10022, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:01,281 INFO [Listener at localhost/38001] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-08-09 04:56:01,281 DEBUG [Listener at localhost/38001] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-08-09 04:56:01,283 INFO [Listener at localhost/38001] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-09 04:56:01,284 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:01,284 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:01,284 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:56:01,284 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:01,284 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:56:01,284 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:56:01,285 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43063 2023-08-09 04:56:01,286 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:01,287 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:01,288 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43063 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:01,292 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:430630x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:01,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43063-0x101e814d6e1002a connected 2023-08-09 04:56:01,297 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:56:01,298 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:56:01,298 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-09 04:56:01,303 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43063 2023-08-09 04:56:01,303 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43063 2023-08-09 04:56:01,303 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43063 2023-08-09 04:56:01,305 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43063 2023-08-09 04:56:01,305 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43063 2023-08-09 04:56:01,305 INFO [Listener at localhost/38001] master.HMaster(444): hbase.rootdir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1, hbase.cluster.distributed=false 2023-08-09 04:56:01,306 DEBUG [pool-560-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-08-09 04:56:01,320 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-09 04:56:01,320 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:01,320 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:01,321 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:56:01,321 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:01,321 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:56:01,321 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:56:01,321 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42717 2023-08-09 04:56:01,322 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:56:01,326 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:56:01,326 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:01,327 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:01,328 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42717 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:01,330 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:427170x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:01,333 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:427170x0, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:56:01,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42717-0x101e814d6e1002b connected 2023-08-09 04:56:01,334 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:56:01,334 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-09 04:56:01,336 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42717 2023-08-09 04:56:01,337 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42717 2023-08-09 04:56:01,337 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42717 2023-08-09 04:56:01,341 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42717 2023-08-09 04:56:01,341 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42717 2023-08-09 04:56:01,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,353 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-09 04:56:01,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,355 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-09 04:56:01,355 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-09 04:56:01,355 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:56:01,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-09 04:56:01,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-09 04:56:01,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,43063,1691556961283 from backup master directory 2023-08-09 04:56:01,359 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,359 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-09 04:56:01,359 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:56:01,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:01,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x67640528 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:01,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73349a79, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:01,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:56:01,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-09 04:56:01,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:01,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:01,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543 to hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543-dead as it is dead 2023-08-09 04:56:01,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543-dead/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 2023-08-09 04:56:01,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543-dead/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 after 0ms 2023-08-09 04:56:01,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543-dead/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 to hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 2023-08-09 04:56:01,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,38065,1691556930543-dead 2023-08-09 04:56:01,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:01,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43063%2C1691556961283.jenkins-hbase4.apache.org%2C43063%2C1691556961283.regiongroup-0, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,43063,1691556961283, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/oldWALs, maxLogs=10 2023-08-09 04:56:01,441 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:56:01,441 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:56:01,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,43063,1691556961283/jenkins-hbase4.apache.org%2C43063%2C1691556961283.jenkins-hbase4.apache.org%2C43063%2C1691556961283.regiongroup-0.1691556961422 2023-08-09 04:56:01,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK], DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK]] 2023-08-09 04:56:01,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:01,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:01,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:01,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:01,446 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:01,447 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-09 04:56:01,447 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-09 04:56:01,454 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/cf9c8330e63f4d86a56e6426f27a9048 2023-08-09 04:56:01,454 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:01,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-09 04:56:01,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 2023-08-09 04:56:01,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 2023-08-09 04:56:01,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38065%2C1691556930543.jenkins-hbase4.apache.org%2C38065%2C1691556930543.regiongroup-0.1691556932417 2023-08-09 04:56:01,466 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:01,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-08-09 04:56:01,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64731058, jitterRate=-0.03543207049369812}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:56:01,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:56:01,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-09 04:56:01,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-09 04:56:01,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-09 04:56:01,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-09 04:56:01,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-09 04:56:01,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-09 04:56:01,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-09 04:56:01,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-09 04:56:01,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-09 04:56:01,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-09 04:56:01,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-09 04:56:01,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-08-09 04:56:01,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-09 04:56:01,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-09 04:56:01,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-09 04:56:01,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42251,1691556931838, table=hbase:meta, region=1588230740 2023-08-09 04:56:01,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-08-09 04:56:01,490 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42251,1691556931838 already deleted, retry=false 2023-08-09 04:56:01,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42251,1691556931838 on jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42251,1691556931838, splitWal=true, meta=true 2023-08-09 04:56:01,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,42251,1691556931838 (carryingMeta=true) jenkins-hbase4.apache.org,42251,1691556931838/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1de11c42[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:56:01,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,41117,1691556931790 already deleted, retry=false 2023-08-09 04:56:01,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,41117,1691556931790 on jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,41117,1691556931790, splitWal=true, meta=false 2023-08-09 04:56:01,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,41117,1691556931790 (carryingMeta=false) jenkins-hbase4.apache.org,41117,1691556931790/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@5c77b834[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:56:01,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-09 04:56:01,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-09 04:56:01,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-08-09 04:56:01,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-09 04:56:01,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-08-09 04:56:01,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-08-09 04:56:01,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-08-09 04:56:01,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-08-09 04:56:01,510 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-09 04:56:01,510 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-09 04:56:01,510 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:56:01,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,43063,1691556961283, sessionid=0x101e814d6e1002a, setting cluster-up flag (Was=false) 2023-08-09 04:56:01,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-08-09 04:56:01,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-08-09 04:56:01,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:01,519 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/.hbase-snapshot/.tmp 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:56:01,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691556991522 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-09 04:56:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-09 04:56:01,525 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41117,1691556931790; numProcessing=1 2023-08-09 04:56:01,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-09 04:56:01,525 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41117,1691556931790, splitWal=true, meta=false 2023-08-09 04:56:01,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-09 04:56:01,525 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42251,1691556931838; numProcessing=2 2023-08-09 04:56:01,525 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42251,1691556931838, splitWal=true, meta=true 2023-08-09 04:56:01,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-09 04:56:01,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-09 04:56:01,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556961525,5,FailOnTimeoutGroup] 2023-08-09 04:56:01,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556961525,5,FailOnTimeoutGroup] 2023-08-09 04:56:01,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,526 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42251,1691556931838, splitWal=true, meta=true, isMeta: true 2023-08-09 04:56:01,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-09 04:56:01,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691556961526, completionTime=-1 2023-08-09 04:56:01,527 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-09 04:56:01,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-09 04:56:01,528 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838-splitting 2023-08-09 04:56:01,528 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838-splitting dir is empty, no logs to split. 2023-08-09 04:56:01,528 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42251,1691556931838 WAL count=0, meta=true 2023-08-09 04:56:01,530 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838-splitting dir is empty, no logs to split. 2023-08-09 04:56:01,530 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42251,1691556931838 WAL count=0, meta=true 2023-08-09 04:56:01,530 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42251,1691556931838 WAL splitting is done? wals=0, meta=true 2023-08-09 04:56:01,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-09 04:56:01,532 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-09 04:56:01,534 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:01,543 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(951): ClusterId : 2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 2023-08-09 04:56:01,543 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:56:01,545 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:56:01,545 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:56:01,548 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:56:01,550 DEBUG [RS:0;jenkins-hbase4:42717] zookeeper.ReadOnlyZKClient(139): Connect 0x156c9911 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:01,553 DEBUG [RS:0;jenkins-hbase4:42717] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bab65cc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:01,553 DEBUG [RS:0;jenkins-hbase4:42717] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f94f261, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:56:01,562 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42717 2023-08-09 04:56:01,562 INFO [RS:0;jenkins-hbase4:42717] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:56:01,562 INFO [RS:0;jenkins-hbase4:42717] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:56:01,562 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:56:01,562 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43063,1691556961283 with isa=jenkins-hbase4.apache.org/172.31.14.131:42717, startcode=1691556961320 2023-08-09 04:56:01,562 DEBUG [RS:0;jenkins-hbase4:42717] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:56:01,564 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44251, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:56:01,565 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43063] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,565 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1 2023-08-09 04:56:01,565 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45825 2023-08-09 04:56:01,566 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:56:01,567 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:56:01,567 DEBUG [RS:0;jenkins-hbase4:42717] zookeeper.ZKUtil(162): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,568 WARN [RS:0;jenkins-hbase4:42717] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:56:01,568 INFO [RS:0;jenkins-hbase4:42717] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:01,568 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:01,568 INFO [RS:0;jenkins-hbase4:42717] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:01,569 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-09 04:56:01,583 INFO [RS:0;jenkins-hbase4:42717] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-09 04:56:01,583 DEBUG [RS:0;jenkins-hbase4:42717] zookeeper.ZKUtil(162): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,584 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:56:01,584 INFO [RS:0;jenkins-hbase4:42717] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:56:01,585 INFO [RS:0;jenkins-hbase4:42717] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:56:01,586 INFO [RS:0;jenkins-hbase4:42717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:56:01,586 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,586 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:56:01,587 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,588 DEBUG [RS:0;jenkins-hbase4:42717] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:01,589 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,589 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,589 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,600 INFO [RS:0;jenkins-hbase4:42717] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:56:01,600 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42717,1691556961320-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:01,611 INFO [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:56:01,612 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1691556950935.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:01,622 INFO [RS:0;jenkins-hbase4:42717] regionserver.Replication(203): jenkins-hbase4.apache.org,42717,1691556961320 started 2023-08-09 04:56:01,622 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42717,1691556961320, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42717, sessionid=0x101e814d6e1002b 2023-08-09 04:56:01,622 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42717,1691556961320' 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42717,1691556961320' 2023-08-09 04:56:01,623 DEBUG [RS:0;jenkins-hbase4:42717] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-09 04:56:01,624 DEBUG [RS:0;jenkins-hbase4:42717] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-09 04:56:01,624 DEBUG [RS:0;jenkins-hbase4:42717] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:56:01,624 INFO [RS:0;jenkins-hbase4:42717] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:56:01,624 INFO [RS:0;jenkins-hbase4:42717] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:56:01,633 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x499e3c19 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:01,637 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7e008043, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:01,637 INFO [RS:0;jenkins-hbase4:42717.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:01,640 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:01,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e1002f connected 2023-08-09 04:56:01,643 INFO [RS:0;jenkins-hbase4:42717.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:56:01,684 DEBUG [jenkins-hbase4:43063] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-09 04:56:01,685 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42717,1691556961320, state=OPENING 2023-08-09 04:56:01,686 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-09 04:56:01,687 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-09 04:56:01,687 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42717,1691556961320}] 2023-08-09 04:56:01,724 INFO [RS:0;jenkins-hbase4:42717] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:01,726 INFO [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:56:01,742 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:56:01,742 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:56:01,750 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0 for peer 1 2023-08-09 04:56:01,750 INFO [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 2023-08-09 04:56:01,750 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0 2023-08-09 04:56:01,751 INFO [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:56:01,751 DEBUG [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK], DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK]] 2023-08-09 04:56:01,751 INFO [RS:0;jenkins-hbase4:42717.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0 2023-08-09 04:56:01,755 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:01,844 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:01,844 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:56:01,845 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58742, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:56:01,849 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-09 04:56:01,849 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:01,849 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:01,849 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:01,851 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42717%2C1691556961320.meta, suffix=.meta, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:56:01,860 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:01,865 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:56:01,865 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:56:01,867 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.meta.1691556961851.meta 2023-08-09 04:56:01,867 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK], DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK]] 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-09 04:56:01,868 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-09 04:56:01,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-09 04:56:01,871 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:56:01,872 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info 2023-08-09 04:56:01,872 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info 2023-08-09 04:56:01,872 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:56:01,880 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info/ef3f847fb4f644f68aaa332e672d35f4 2023-08-09 04:56:01,881 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:01,881 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:56:01,882 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:56:01,882 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:56:01,882 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:56:01,889 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier/42fdd228bc2447059dcd2931fd60c30e 2023-08-09 04:56:01,890 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:01,890 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:56:01,891 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table 2023-08-09 04:56:01,891 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table 2023-08-09 04:56:01,891 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:56:01,897 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/table/3a83a2e674af4ac38a084dae47afdf21 2023-08-09 04:56:01,897 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:01,897 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740 2023-08-09 04:56:01,898 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740 2023-08-09 04:56:01,901 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:56:01,902 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:56:01,902 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59271638, jitterRate=-0.11678376793861389}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:56:01,902 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:56:01,903 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1691556961844 2023-08-09 04:56:01,908 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-09 04:56:01,909 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-09 04:56:01,909 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42717,1691556961320, state=OPEN 2023-08-09 04:56:01,910 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-09 04:56:01,911 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-09 04:56:01,912 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-08-09 04:56:01,912 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42717,1691556961320 in 224 msec 2023-08-09 04:56:01,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=18 2023-08-09 04:56:01,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 381 msec 2023-08-09 04:56:01,965 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,070 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,175 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,284 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,388 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,493 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,607 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,712 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,817 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:02,922 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,027 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,079 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1552ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-08-09 04:56:03,132 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,238 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,343 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,448 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,553 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,658 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,763 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,868 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:03,973 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,078 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,182 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,287 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,392 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,496 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-08-09 04:56:04,600 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,705 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,809 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:04,829 INFO [jenkins-hbase4:45045Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:56:04,829 INFO [jenkins-hbase4:45045Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-09 04:56:04,914 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,019 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,125 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,229 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,335 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,439 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,545 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,650 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,754 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,858 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:05,963 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:06,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-09 04:56:06,034 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-09 04:56:06,038 DEBUG [hconnection-0x395a6c4e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:06,039 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:56:06,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=cb1d40dc2c341948bac0897d809dc4a0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41117,1691556931790, regionLocation=jenkins-hbase4.apache.org,41117,1691556931790, openSeqNum=2 2023-08-09 04:56:06,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=4d1b412f72764eba378319d319bc7848, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42251,1691556931838, regionLocation=jenkins-hbase4.apache.org,42251,1691556931838, openSeqNum=2 2023-08-09 04:56:06,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=2ccff9a5354735fa815894d8a555d598, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42251,1691556931838, regionLocation=jenkins-hbase4.apache.org,42251,1691556931838, openSeqNum=2 2023-08-09 04:56:06,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-09 04:56:06,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691557026046 2023-08-09 04:56:06,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691557086046 2023-08-09 04:56:06,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-08-09 04:56:06,062 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42251,1691556931838 had 3 regions 2023-08-09 04:56:06,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43063,1691556961283-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:06,062 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,41117,1691556931790 had 1 regions 2023-08-09 04:56:06,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43063,1691556961283-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:06,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43063,1691556961283-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:06,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43063, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:06,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:06,063 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. is NOT online; state={cb1d40dc2c341948bac0897d809dc4a0 state=OPEN, ts=1691556966045, server=jenkins-hbase4.apache.org,41117,1691556931790}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-09 04:56:06,064 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42251,1691556931838, splitWal=true, meta=true, isMeta: false 2023-08-09 04:56:06,064 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41117,1691556931790, splitWal=true, meta=false, isMeta: false 2023-08-09 04:56:06,068 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,41117,1691556931790/hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0., unknown_server=jenkins-hbase4.apache.org,42251,1691556931838/t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848., unknown_server=jenkins-hbase4.apache.org,42251,1691556931838/t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:06,068 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838-splitting dir is empty, no logs to split. 2023-08-09 04:56:06,068 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,42251,1691556931838 WAL count=0, meta=false 2023-08-09 04:56:06,068 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790-splitting 2023-08-09 04:56:06,070 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790-splitting dir is empty, no logs to split. 2023-08-09 04:56:06,070 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,41117,1691556931790 WAL count=0, meta=false 2023-08-09 04:56:06,071 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42251,1691556931838-splitting dir is empty, no logs to split. 2023-08-09 04:56:06,071 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:06,071 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,42251,1691556931838 WAL count=0, meta=false 2023-08-09 04:56:06,071 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42251,1691556931838 WAL splitting is done? wals=0, meta=false 2023-08-09 04:56:06,073 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790-splitting dir is empty, no logs to split. 2023-08-09 04:56:06,073 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN}, {pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN}] 2023-08-09 04:56:06,073 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,41117,1691556931790 WAL count=0, meta=false 2023-08-09 04:56:06,073 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,41117,1691556931790 WAL splitting is done? wals=0, meta=false 2023-08-09 04:56:06,074 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN 2023-08-09 04:56:06,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN 2023-08-09 04:56:06,075 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:06,075 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:06,075 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,41117,1691556931790 failed, ignore...File hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,41117,1691556931790-splitting does not exist. 2023-08-09 04:56:06,076 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN}] 2023-08-09 04:56:06,077 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN 2023-08-09 04:56:06,077 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:06,077 DEBUG [jenkins-hbase4:43063] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-09 04:56:06,080 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=2ccff9a5354735fa815894d8a555d598, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:06,080 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=4d1b412f72764eba378319d319bc7848, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:06,080 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=cb1d40dc2c341948bac0897d809dc4a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:06,080 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556966080"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556966080"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556966080"}]},"ts":"1691556966080"} 2023-08-09 04:56:06,080 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556966080"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556966080"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556966080"}]},"ts":"1691556966080"} 2023-08-09 04:56:06,080 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556966080"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556966080"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556966080"}]},"ts":"1691556966080"} 2023-08-09 04:56:06,083 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 2ccff9a5354735fa815894d8a555d598, server=jenkins-hbase4.apache.org,42717,1691556961320}] 2023-08-09 04:56:06,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 4d1b412f72764eba378319d319bc7848, server=jenkins-hbase4.apache.org,42717,1691556961320}] 2023-08-09 04:56:06,084 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure cb1d40dc2c341948bac0897d809dc4a0, server=jenkins-hbase4.apache.org,42717,1691556961320}] 2023-08-09 04:56:06,176 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:06,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:06,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:06,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1, suffix=, logDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320, archiveDir=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs, maxLogs=10 2023-08-09 04:56:06,240 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:06,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cb1d40dc2c341948bac0897d809dc4a0, NAME => 'hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:06,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:06,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,242 INFO [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,244 DEBUG [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info 2023-08-09 04:56:06,244 DEBUG [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info 2023-08-09 04:56:06,244 INFO [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb1d40dc2c341948bac0897d809dc4a0 columnFamilyName info 2023-08-09 04:56:06,251 DEBUG [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/info/e2b6b9e74b5f418ca048abb7dca929af 2023-08-09 04:56:06,251 INFO [StoreOpener-cb1d40dc2c341948bac0897d809dc4a0-1] regionserver.HStore(310): Store=cb1d40dc2c341948bac0897d809dc4a0/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:06,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,253 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,255 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK] 2023-08-09 04:56:06,255 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK] 2023-08-09 04:56:06,257 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:06,258 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cb1d40dc2c341948bac0897d809dc4a0; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68137839, jitterRate=0.015332922339439392}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:56:06,258 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cb1d40dc2c341948bac0897d809dc4a0: 2023-08-09 04:56:06,259 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0., pid=27, masterSystemTime=1691556966234 2023-08-09 04:56:06,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1 for peer 1 2023-08-09 04:56:06,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240 2023-08-09 04:56:06,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1 2023-08-09 04:56:06,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:56:06,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38063,DS-9193ffdc-8531-4e08-9163-ff0146654969,DISK], DatanodeInfoWithStorage[127.0.0.1:46353,DS-25f09e9e-adae-4cf2-bec8-46586d71f914,DISK]] 2023-08-09 04:56:06,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1 2023-08-09 04:56:06,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2ccff9a5354735fa815894d8a555d598, NAME => 't2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:06,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:06,263 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:06,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,263 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:06,263 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=cb1d40dc2c341948bac0897d809dc4a0, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:06,263 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556966263"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556966263"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556966263"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556966263"}]},"ts":"1691556966263"} 2023-08-09 04:56:06,266 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,267 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1 2023-08-09 04:56:06,267 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1 2023-08-09 04:56:06,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-08-09 04:56:06,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure cb1d40dc2c341948bac0897d809dc4a0, server=jenkins-hbase4.apache.org,42717,1691556961320 in 181 msec 2023-08-09 04:56:06,267 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ccff9a5354735fa815894d8a555d598 columnFamilyName cf1 2023-08-09 04:56:06,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-09 04:56:06,269 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=19 2023-08-09 04:56:06,269 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=cb1d40dc2c341948bac0897d809dc4a0, ASSIGN in 191 msec 2023-08-09 04:56:06,269 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,41117,1691556931790 after splitting done 2023-08-09 04:56:06,269 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,41117,1691556931790 from processing; numProcessing=1 2023-08-09 04:56:06,271 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,41117,1691556931790, splitWal=true, meta=false in 4.7770 sec 2023-08-09 04:56:06,277 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7798f4cf09d8411fbc9c9641fbb654f8 2023-08-09 04:56:06,278 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1/7798f4cf09d8411fbc9c9641fbb654f8 2023-08-09 04:56:06,278 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(310): Store=2ccff9a5354735fa815894d8a555d598/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:06,278 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,279 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep 2023-08-09 04:56:06,279 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep 2023-08-09 04:56:06,280 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ccff9a5354735fa815894d8a555d598 columnFamilyName norep 2023-08-09 04:56:06,280 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,280 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:56:06,280 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:56:06,280 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,285 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eed3659271f644c2a30750a1a1bf1f78 2023-08-09 04:56:06,285 DEBUG [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep/eed3659271f644c2a30750a1a1bf1f78 2023-08-09 04:56:06,285 INFO [StoreOpener-2ccff9a5354735fa815894d8a555d598-1] regionserver.HStore(310): Store=2ccff9a5354735fa815894d8a555d598/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:06,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:56:06,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:06,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2ccff9a5354735fa815894d8a555d598; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73521352, jitterRate=0.09555351734161377}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:56:06,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2ccff9a5354735fa815894d8a555d598: 2023-08-09 04:56:06,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598., pid=25, masterSystemTime=1691556966234 2023-08-09 04:56:06,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:06,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:06,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:06,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4d1b412f72764eba378319d319bc7848, NAME => 't1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:06,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:06,295 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=2ccff9a5354735fa815894d8a555d598, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:06,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,295 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556966295"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556966295"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556966295"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556966295"}]},"ts":"1691556966295"} 2023-08-09 04:56:06,297 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,298 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1 2023-08-09 04:56:06,298 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1 2023-08-09 04:56:06,298 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4d1b412f72764eba378319d319bc7848 columnFamilyName cf1 2023-08-09 04:56:06,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-08-09 04:56:06,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 2ccff9a5354735fa815894d8a555d598, server=jenkins-hbase4.apache.org,42717,1691556961320 in 214 msec 2023-08-09 04:56:06,300 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=2ccff9a5354735fa815894d8a555d598, ASSIGN in 226 msec 2023-08-09 04:56:06,305 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0860b4b531b648f5bdf2c07ea6e9cf3a 2023-08-09 04:56:06,305 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1/0860b4b531b648f5bdf2c07ea6e9cf3a 2023-08-09 04:56:06,306 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(310): Store=4d1b412f72764eba378319d319bc7848/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:06,306 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,307 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep 2023-08-09 04:56:06,307 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep 2023-08-09 04:56:06,307 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4d1b412f72764eba378319d319bc7848 columnFamilyName norep 2023-08-09 04:56:06,316 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e77efce3e5ac4e678334d3d7285a6da5 2023-08-09 04:56:06,316 DEBUG [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(539): loaded hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep/e77efce3e5ac4e678334d3d7285a6da5 2023-08-09 04:56:06,316 INFO [StoreOpener-4d1b412f72764eba378319d319bc7848-1] regionserver.HStore(310): Store=4d1b412f72764eba378319d319bc7848/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:06,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:56:06,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:06,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4d1b412f72764eba378319d319bc7848; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72875674, jitterRate=0.0859321653842926}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:56:06,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4d1b412f72764eba378319d319bc7848: 2023-08-09 04:56:06,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848., pid=26, masterSystemTime=1691556966234 2023-08-09 04:56:06,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:06,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:06,325 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=4d1b412f72764eba378319d319bc7848, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:06,325 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556966325"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556966325"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556966325"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556966325"}]},"ts":"1691556966325"} 2023-08-09 04:56:06,328 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-08-09 04:56:06,328 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 4d1b412f72764eba378319d319bc7848, server=jenkins-hbase4.apache.org,42717,1691556961320 in 243 msec 2023-08-09 04:56:06,330 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=18 2023-08-09 04:56:06,330 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42251,1691556931838 after splitting done 2023-08-09 04:56:06,330 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=4d1b412f72764eba378319d319bc7848, ASSIGN in 255 msec 2023-08-09 04:56:06,330 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42251,1691556931838 from processing; numProcessing=0 2023-08-09 04:56:06,331 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42251,1691556931838, splitWal=true, meta=true in 4.8400 sec 2023-08-09 04:56:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:56:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240, lastWalPosition=823, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:56:06,385 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,489 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,570 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:56:06,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,593 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,622 INFO [jenkins-hbase4:42717Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0]: currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 at position: 422 walGroup [jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1]: currently replicating from: hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240 at position: 823 2023-08-09 04:56:06,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,698 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,803 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:06,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:06,907 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,011 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-08-09 04:56:07,078 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-08-09 04:56:07,082 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-08-09 04:56:07,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.723sec 2023-08-09 04:56:07,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-09 04:56:07,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-09 04:56:07,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-09 04:56:07,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43063,1691556961283-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-09 04:56:07,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43063,1691556961283-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-09 04:56:07,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-09 04:56:07,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,115 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,148 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x1c47558d to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:07,153 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27464c26, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:07,154 DEBUG [hconnection-0x4d844808-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:07,156 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58746, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:56:07,162 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-09 04:56:07,162 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1c47558d to 127.0.0.1:63198 2023-08-09 04:56:07,162 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,163 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:43063 after: jenkins-hbase4.apache.org:43063 2023-08-09 04:56:07,164 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x12cce40a to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:07,169 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7de72a53, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:07,169 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x210e00dc to 127.0.0.1:63198 2023-08-09 04:56:07,169 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,169 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-09 04:56:07,170 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(257): Found active master hash=643082231, stopped=false 2023-08-09 04:56:07,170 INFO [Listener at localhost/38001] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:56:07,172 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:56:07,172 INFO [Listener at localhost/38001] procedure2.ProcedureExecutor(629): Stopping 2023-08-09 04:56:07,172 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:56:07,173 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5df54625 to 127.0.0.1:63198 2023-08-09 04:56:07,174 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,175 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:56:07,176 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45045,1691556944435' ***** 2023-08-09 04:56:07,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:56:07,176 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:56:07,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:56:07,176 INFO [RS:0;jenkins-hbase4:45045] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:56:07,176 INFO [RS:0;jenkins-hbase4:45045] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:56:07,176 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:56:07,176 INFO [RS:0;jenkins-hbase4:45045] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:56:07,177 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(3305): Received CLOSE for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:07,177 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(3305): Received CLOSE for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:07,177 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(3305): Received CLOSE for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:07,177 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:56:07,178 DEBUG [RS:0;jenkins-hbase4:45045] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25e5c999 to 127.0.0.1:63198 2023-08-09 04:56:07,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36954931527a871a4ac33b21c04863f3, disabling compactions & flushes 2023-08-09 04:56:07,178 DEBUG [RS:0;jenkins-hbase4:45045] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:07,178 INFO [RS:0;jenkins-hbase4:45045] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:56:07,178 INFO [RS:0;jenkins-hbase4:45045] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:56:07,178 INFO [RS:0;jenkins-hbase4:45045] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:56:07,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:07,178 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-09 04:56:07,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. after waiting 0 ms 2023-08-09 04:56:07,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:07,179 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-09 04:56:07,179 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1478): Online Regions={36954931527a871a4ac33b21c04863f3=hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., baf34b839cf96847099b70ccf7c3d7ea=t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., 1588230740=hbase:meta,,1.1588230740, 629be257c9d8450abcf2763c66f8a37a=t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.} 2023-08-09 04:56:07,180 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:56:07,180 DEBUG [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1504): Waiting on 1588230740, 36954931527a871a4ac33b21c04863f3, 629be257c9d8450abcf2763c66f8a37a, baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:07,180 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:56:07,180 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:56:07,180 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:56:07,180 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:56:07,181 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-08-09 04:56:07,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-08-09 04:56:07,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:07,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:56:07,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:07,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing baf34b839cf96847099b70ccf7c3d7ea, disabling compactions & flushes 2023-08-09 04:56:07,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:07,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:07,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. after waiting 0 ms 2023-08-09 04:56:07,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:07,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing baf34b839cf96847099b70ccf7c3d7ea 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-08-09 04:56:07,213 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/info/6bbf0f789adf465aa07c7a710d6dbf92 2023-08-09 04:56:07,222 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/info/6bbf0f789adf465aa07c7a710d6dbf92 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/6bbf0f789adf465aa07c7a710d6dbf92 2023-08-09 04:56:07,223 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:56:07,229 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,236 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/6bbf0f789adf465aa07c7a710d6dbf92, entries=24, sequenceid=31, filesize=7.4 K 2023-08-09 04:56:07,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/.tmp/cf1/05caab739a2340a0b24d14c800479d1a 2023-08-09 04:56:07,237 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 57ms, sequenceid=31, compaction requested=false 2023-08-09 04:56:07,251 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-08-09 04:56:07,251 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-09 04:56:07,252 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:56:07,252 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:56:07,252 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-09 04:56:07,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05caab739a2340a0b24d14c800479d1a 2023-08-09 04:56:07,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/.tmp/cf1/05caab739a2340a0b24d14c800479d1a as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/05caab739a2340a0b24d14c800479d1a 2023-08-09 04:56:07,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05caab739a2340a0b24d14c800479d1a 2023-08-09 04:56:07,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/05caab739a2340a0b24d14c800479d1a, entries=50, sequenceid=14, filesize=6.4 K 2023-08-09 04:56:07,259 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for baf34b839cf96847099b70ccf7c3d7ea in 63ms, sequenceid=14, compaction requested=false 2023-08-09 04:56:07,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-08-09 04:56:07,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:07,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:56:07,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:07,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 629be257c9d8450abcf2763c66f8a37a, disabling compactions & flushes 2023-08-09 04:56:07,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:07,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:07,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. after waiting 0 ms 2023-08-09 04:56:07,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:07,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 629be257c9d8450abcf2763c66f8a37a 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-08-09 04:56:07,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/.tmp/cf1/625d721f48ff4ff6ad709109dff79fcb 2023-08-09 04:56:07,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 625d721f48ff4ff6ad709109dff79fcb 2023-08-09 04:56:07,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/.tmp/cf1/625d721f48ff4ff6ad709109dff79fcb as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/625d721f48ff4ff6ad709109dff79fcb 2023-08-09 04:56:07,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 625d721f48ff4ff6ad709109dff79fcb 2023-08-09 04:56:07,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/625d721f48ff4ff6ad709109dff79fcb, entries=100, sequenceid=15, filesize=7.9 K 2023-08-09 04:56:07,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 629be257c9d8450abcf2763c66f8a37a in 43ms, sequenceid=15, compaction requested=false 2023-08-09 04:56:07,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-08-09 04:56:07,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:07,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:56:07,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:07,333 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,380 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45045,1691556944435; all regions closed. 2023-08-09 04:56:07,390 DEBUG [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:56:07,390 INFO [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45045%2C1691556944435.meta:.meta(num 1691556945042) 2023-08-09 04:56:07,396 DEBUG [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:56:07,397 INFO [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45045%2C1691556944435.jenkins-hbase4.apache.org%2C45045%2C1691556944435.regiongroup-0:(num 1691556944934) 2023-08-09 04:56:07,401 DEBUG [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:56:07,401 INFO [RS:0;jenkins-hbase4:45045] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45045%2C1691556944435.jenkins-hbase4.apache.org%2C45045%2C1691556944435.regiongroup-1:(num 1691556949446) 2023-08-09 04:56:07,401 DEBUG [RS:0;jenkins-hbase4:45045] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,401 INFO [RS:0;jenkins-hbase4:45045] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:56:07,401 INFO [RS:0;jenkins-hbase4:45045] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-09 04:56:07,401 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:56:07,401 DEBUG [RS:0;jenkins-hbase4:45045] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6011c76c to 127.0.0.1:63198 2023-08-09 04:56:07,402 DEBUG [RS:0;jenkins-hbase4:45045] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,403 INFO [RS:0;jenkins-hbase4:45045] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45045 2023-08-09 04:56:07,409 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45045,1691556944435 2023-08-09 04:56:07,409 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:07,409 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:07,409 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10026, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:07,409 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:56:07,410 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45045,1691556944435] 2023-08-09 04:56:07,410 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45045,1691556944435; numProcessing=1 2023-08-09 04:56:07,411 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45045,1691556944435 already deleted, retry=false 2023-08-09 04:56:07,411 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45045,1691556944435 expired; onlineServers=0 2023-08-09 04:56:07,411 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42053,1691556944384' ***** 2023-08-09 04:56:07,411 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-09 04:56:07,412 DEBUG [M:0;jenkins-hbase4:42053] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@226b27e5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:56:07,412 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42053,1691556944384 2023-08-09 04:56:07,412 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42053,1691556944384; all regions closed. 2023-08-09 04:56:07,412 DEBUG [M:0;jenkins-hbase4:42053] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:07,412 DEBUG [M:0;jenkins-hbase4:42053] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-09 04:56:07,412 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-09 04:56:07,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556944711] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556944711,5,FailOnTimeoutGroup] 2023-08-09 04:56:07,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556944711] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556944711,5,FailOnTimeoutGroup] 2023-08-09 04:56:07,412 DEBUG [M:0;jenkins-hbase4:42053] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-09 04:56:07,413 INFO [M:0;jenkins-hbase4:42053] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-09 04:56:07,413 INFO [M:0;jenkins-hbase4:42053] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-09 04:56:07,414 INFO [M:0;jenkins-hbase4:42053] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-09 04:56:07,414 DEBUG [M:0;jenkins-hbase4:42053] master.HMaster(1512): Stopping service threads 2023-08-09 04:56:07,414 INFO [M:0;jenkins-hbase4:42053] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-09 04:56:07,414 ERROR [M:0;jenkins-hbase4:42053] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-09 04:56:07,414 INFO [M:0;jenkins-hbase4:42053] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-09 04:56:07,415 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-09 04:56:07,415 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-09 04:56:07,415 DEBUG [M:0;jenkins-hbase4:42053] zookeeper.ZKUtil(398): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-09 04:56:07,415 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:56:07,415 WARN [M:0;jenkins-hbase4:42053] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-09 04:56:07,415 INFO [M:0;jenkins-hbase4:42053] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-09 04:56:07,416 INFO [M:0;jenkins-hbase4:42053] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-09 04:56:07,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:56:07,417 DEBUG [M:0;jenkins-hbase4:42053] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:56:07,417 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:07,417 DEBUG [M:0;jenkins-hbase4:42053] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:07,417 DEBUG [M:0;jenkins-hbase4:42053] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:56:07,417 DEBUG [M:0;jenkins-hbase4:42053] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:07,417 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=35.65 KB heapSize=44.37 KB 2023-08-09 04:56:07,432 INFO [M:0;jenkins-hbase4:42053] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=35.65 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b4e9a3dd7a57490a83c9b47baa094b5e 2023-08-09 04:56:07,437 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,438 DEBUG [M:0;jenkins-hbase4:42053] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b4e9a3dd7a57490a83c9b47baa094b5e as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b4e9a3dd7a57490a83c9b47baa094b5e 2023-08-09 04:56:07,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,444 INFO [M:0;jenkins-hbase4:42053] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b4e9a3dd7a57490a83c9b47baa094b5e, entries=12, sequenceid=218, filesize=6.4 K 2023-08-09 04:56:07,445 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegion(2948): Finished flush of dataSize ~35.65 KB/36504, heapSize ~44.35 KB/45416, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=218, compaction requested=false 2023-08-09 04:56:07,447 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:07,447 DEBUG [M:0;jenkins-hbase4:42053] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:56:07,451 INFO [M:0;jenkins-hbase4:42053] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-09 04:56:07,451 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:56:07,452 INFO [M:0;jenkins-hbase4:42053] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42053 2023-08-09 04:56:07,457 DEBUG [M:0;jenkins-hbase4:42053] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,42053,1691556944384 already deleted, retry=false 2023-08-09 04:56:07,542 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,566 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:07,566 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:42053-0x101e814d6e1001c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:07,566 INFO [M:0;jenkins-hbase4:42053] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42053,1691556944384; zookeeper connection closed. 2023-08-09 04:56:07,648 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-09 04:56:07,667 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:07,667 INFO [RS:0;jenkins-hbase4:45045] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45045,1691556944435; zookeeper connection closed. 2023-08-09 04:56:07,667 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:45045-0x101e814d6e1001d, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:07,668 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3ac9c776] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3ac9c776 2023-08-09 04:56:07,668 INFO [Listener at localhost/38001] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-09 04:56:07,670 DEBUG [hconnection-0x47074a73-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:07,672 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58752, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:56:07,760 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9903 2023-08-09 04:56:07,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 86 WAL entries eligible for replication 2023-08-09 04:56:07,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9903 2023-08-09 04:56:07,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/112=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/113=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/114=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/115=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/116=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/117=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/118=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/119=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/120=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/121=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/122=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/123=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/124=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/125=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/126=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/127=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/128=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/129=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/130=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/131=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/132=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/133=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/134=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/135=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/136=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/137=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/138=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/139=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/140=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/141=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/142=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/143=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/144=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/145=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/146=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/147=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/148=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/149=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/150=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/151=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/152=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/153=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/154=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/155=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/156=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/157=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/158=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/159=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/160=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/161=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/162=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/163=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/164=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/165=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/166=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/167=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/168=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/169=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/170=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/171=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/172=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/173=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/174=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/175=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/176=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/177=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/178=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/179=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/180=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/181=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/182=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/183=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/184=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/185=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/186=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/187=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/188=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/189=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/190=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/191=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/192=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/193=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/194=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/195=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/196=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/197=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240, lastWalPosition=9903, nbRowKeys=86, nbHFiles=0, heapSize=17888, lastSeqIds={}, endOfFile=false,usedBufferSize=17888] 2023-08-09 04:56:07,777 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:56:07,865 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:07,877 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:56:07,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22511 2023-08-09 04:56:07,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 118 WAL entries eligible for replication 2023-08-09 04:56:07,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22511 2023-08-09 04:56:07,972 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:08,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 32231 2023-08-09 04:56:08,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 90 WAL entries eligible for replication 2023-08-09 04:56:08,039 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12cce40a to 127.0.0.1:63198 2023-08-09 04:56:08,039 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:08,040 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-09 04:56:08,040 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(257): Found active master hash=1226556134, stopped=false 2023-08-09 04:56:08,040 INFO [Listener at localhost/38001] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:08,042 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-09 04:56:08,042 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-09 04:56:08,042 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:56:08,042 INFO [Listener at localhost/38001] procedure2.ProcedureExecutor(629): Stopping 2023-08-09 04:56:08,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:56:08,043 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67640528 to 127.0.0.1:63198 2023-08-09 04:56:08,043 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:08,043 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42717,1691556961320' ***** 2023-08-09 04:56:08,043 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:56:08,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-09 04:56:08,043 INFO [RS:0;jenkins-hbase4:42717] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:56:08,044 INFO [RS:0;jenkins-hbase4:42717] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:56:08,044 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:56:08,044 INFO [RS:0;jenkins-hbase4:42717] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:56:08,045 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(3305): Received CLOSE for 2ccff9a5354735fa815894d8a555d598 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(3305): Received CLOSE for cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(3305): Received CLOSE for 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:08,047 DEBUG [RS:0;jenkins-hbase4:42717] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x156c9911 to 127.0.0.1:63198 2023-08-09 04:56:08,047 DEBUG [RS:0;jenkins-hbase4:42717] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:56:08,047 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-09 04:56:08,050 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-09 04:56:08,051 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 2ccff9a5354735fa815894d8a555d598=t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598., cb1d40dc2c341948bac0897d809dc4a0=hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0., 4d1b412f72764eba378319d319bc7848=t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848.} 2023-08-09 04:56:08,051 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1504): Waiting on 1588230740, 2ccff9a5354735fa815894d8a555d598, 4d1b412f72764eba378319d319bc7848, cb1d40dc2c341948bac0897d809dc4a0 2023-08-09 04:56:08,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2ccff9a5354735fa815894d8a555d598, disabling compactions & flushes 2023-08-09 04:56:08,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:08,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:08,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. after waiting 0 ms 2023-08-09 04:56:08,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:08,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2ccff9a5354735fa815894d8a555d598 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-08-09 04:56:08,053 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:56:08,053 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:56:08,053 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:56:08,053 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:56:08,053 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:56:08,053 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-08-09 04:56:08,078 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:56:08,080 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:08,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/cf1/85b45c92bea14f92ac1001f17fe078b5 2023-08-09 04:56:08,084 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/info/3f92ccdc71a14042bf2cbd2c826a2bf5 2023-08-09 04:56:08,099 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:56:08,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/norep/1652471973cf4442911a584647591542 2023-08-09 04:56:08,119 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/rep_barrier/c5294e506a594b5396cc434740a3161e 2023-08-09 04:56:08,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/cf1/85b45c92bea14f92ac1001f17fe078b5 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1/85b45c92bea14f92ac1001f17fe078b5 2023-08-09 04:56:08,127 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/info/3f92ccdc71a14042bf2cbd2c826a2bf5 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info/3f92ccdc71a14042bf2cbd2c826a2bf5 2023-08-09 04:56:08,133 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/info/3f92ccdc71a14042bf2cbd2c826a2bf5, entries=24, sequenceid=31, filesize=7.4 K 2023-08-09 04:56:08,134 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/.tmp/rep_barrier/c5294e506a594b5396cc434740a3161e as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier/c5294e506a594b5396cc434740a3161e 2023-08-09 04:56:08,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/cf1/85b45c92bea14f92ac1001f17fe078b5, entries=200, sequenceid=413, filesize=12.5 K 2023-08-09 04:56:08,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/.tmp/norep/1652471973cf4442911a584647591542 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep/1652471973cf4442911a584647591542 2023-08-09 04:56:08,140 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/rep_barrier/c5294e506a594b5396cc434740a3161e, entries=2, sequenceid=31, filesize=4.9 K 2023-08-09 04:56:08,144 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 90ms, sequenceid=31, compaction requested=false 2023-08-09 04:56:08,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/norep/1652471973cf4442911a584647591542, entries=1, sequenceid=413, filesize=4.8 K 2023-08-09 04:56:08,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 2ccff9a5354735fa815894d8a555d598 in 108ms, sequenceid=413, compaction requested=false 2023-08-09 04:56:08,174 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-08-09 04:56:08,178 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-09 04:56:08,178 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:56:08,178 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:56:08,178 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-09 04:56:08,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t2_syncup/2ccff9a5354735fa815894d8a555d598/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-08-09 04:56:08,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:08,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2ccff9a5354735fa815894d8a555d598: 2023-08-09 04:56:08,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691556938403.2ccff9a5354735fa815894d8a555d598. 2023-08-09 04:56:08,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cb1d40dc2c341948bac0897d809dc4a0, disabling compactions & flushes 2023-08-09 04:56:08,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:08,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:08,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. after waiting 0 ms 2023-08-09 04:56:08,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:08,189 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-09 04:56:08,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/hbase/namespace/cb1d40dc2c341948bac0897d809dc4a0/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-08-09 04:56:08,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:08,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cb1d40dc2c341948bac0897d809dc4a0: 2023-08-09 04:56:08,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691556933863.cb1d40dc2c341948bac0897d809dc4a0. 2023-08-09 04:56:08,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4d1b412f72764eba378319d319bc7848, disabling compactions & flushes 2023-08-09 04:56:08,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:08,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:08,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. after waiting 0 ms 2023-08-09 04:56:08,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:08,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4d1b412f72764eba378319d319bc7848 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-08-09 04:56:08,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/cf1/97e2efb13f1a4e4fb0bc748caa2f750c 2023-08-09 04:56:08,251 DEBUG [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1504): Waiting on 4d1b412f72764eba378319d319bc7848 2023-08-09 04:56:08,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/norep/451b735887024989ac29999b9b618323 2023-08-09 04:56:08,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/cf1/97e2efb13f1a4e4fb0bc748caa2f750c as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1/97e2efb13f1a4e4fb0bc748caa2f750c 2023-08-09 04:56:08,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/cf1/97e2efb13f1a4e4fb0bc748caa2f750c, entries=100, sequenceid=213, filesize=8.4 K 2023-08-09 04:56:08,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/.tmp/norep/451b735887024989ac29999b9b618323 as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep/451b735887024989ac29999b9b618323 2023-08-09 04:56:08,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/norep/451b735887024989ac29999b9b618323, entries=1, sequenceid=213, filesize=4.8 K 2023-08-09 04:56:08,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for 4d1b412f72764eba378319d319bc7848 in 89ms, sequenceid=213, compaction requested=false 2023-08-09 04:56:08,300 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-09 04:56:08,300 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-09 04:56:08,301 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-09 04:56:08,301 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727, lastWalPosition=728, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:56:08,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/data/default/t1_syncup/4d1b412f72764eba378319d319bc7848/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-08-09 04:56:08,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:08,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4d1b412f72764eba378319d319bc7848: 2023-08-09 04:56:08,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691556937766.4d1b412f72764eba378319d319bc7848. 2023-08-09 04:56:08,379 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-09 04:56:08,406 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-09 04:56:08,451 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42717,1691556961320; all regions closed. 2023-08-09 04:56:08,458 DEBUG [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:56:08,458 INFO [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42717%2C1691556961320.meta:.meta(num 1691556961851) 2023-08-09 04:56:08,463 DEBUG [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:56:08,463 INFO [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1:(num 1691556966240) 2023-08-09 04:56:08,468 DEBUG [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs 2023-08-09 04:56:08,468 INFO [RS:0;jenkins-hbase4:42717] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0:(num 1691556961727) 2023-08-09 04:56:08,468 DEBUG [RS:0;jenkins-hbase4:42717] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:08,468 INFO [RS:0;jenkins-hbase4:42717] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:56:08,468 INFO [RS:0;jenkins-hbase4:42717] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-09 04:56:08,468 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:56:08,468 INFO [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-09 04:56:08,507 INFO [RS:0;jenkins-hbase4:42717.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/WALs/jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 was moved to hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 2023-08-09 04:56:08,571 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e1002f, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:08,571 DEBUG [RS:0;jenkins-hbase4:42717] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x499e3c19 to 127.0.0.1:63198 2023-08-09 04:56:08,571 DEBUG [RS:0;jenkins-hbase4:42717.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e1002f, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:08,571 DEBUG [RS:0;jenkins-hbase4:42717] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:08,672 INFO [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:42717.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1 terminated 2023-08-09 04:56:08,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(243): Waiting for peers [Source for peer 1]: Interrupted while sleeping between retries 2023-08-09 04:56:08,674 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-08-09 04:56:08,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-08-09 04:56:08,675 INFO [RS:0;jenkins-hbase4:42717] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1 terminated 2023-08-09 04:56:08,676 INFO [RS:0;jenkins-hbase4:42717] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42717 2023-08-09 04:56:08,678 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:08,678 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:56:08,678 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-09 04:56:08,679 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:08,679 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42717,1691556961320; numProcessing=1 2023-08-09 04:56:08,682 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42717,1691556961320 already deleted, retry=false 2023-08-09 04:56:08,682 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42717,1691556961320 expired; onlineServers=0 2023-08-09 04:56:08,682 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43063,1691556961283' ***** 2023-08-09 04:56:08,682 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-09 04:56:08,682 DEBUG [M:0;jenkins-hbase4:43063] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d244f1d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:56:08,683 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43063,1691556961283 2023-08-09 04:56:08,683 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43063,1691556961283; all regions closed. 2023-08-09 04:56:08,683 DEBUG [M:0;jenkins-hbase4:43063] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:08,683 DEBUG [M:0;jenkins-hbase4:43063] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-09 04:56:08,683 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-09 04:56:08,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556961525] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556961525,5,FailOnTimeoutGroup] 2023-08-09 04:56:08,683 DEBUG [M:0;jenkins-hbase4:43063] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-09 04:56:08,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556961525] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556961525,5,FailOnTimeoutGroup] 2023-08-09 04:56:08,684 INFO [M:0;jenkins-hbase4:43063] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-09 04:56:08,685 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-08-09 04:56:08,685 INFO [M:0;jenkins-hbase4:43063] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-09 04:56:08,685 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-09 04:56:08,685 INFO [M:0;jenkins-hbase4:43063] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-09 04:56:08,685 DEBUG [M:0;jenkins-hbase4:43063] master.HMaster(1512): Stopping service threads 2023-08-09 04:56:08,685 INFO [M:0;jenkins-hbase4:43063] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-09 04:56:08,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-09 04:56:08,686 ERROR [M:0;jenkins-hbase4:43063] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-09 04:56:08,686 INFO [M:0;jenkins-hbase4:43063] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-09 04:56:08,686 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-09 04:56:08,687 DEBUG [M:0;jenkins-hbase4:43063] zookeeper.ZKUtil(398): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-08-09 04:56:08,687 WARN [M:0;jenkins-hbase4:43063] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-09 04:56:08,687 INFO [M:0;jenkins-hbase4:43063] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-09 04:56:08,687 INFO [M:0;jenkins-hbase4:43063] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-09 04:56:08,690 DEBUG [M:0;jenkins-hbase4:43063] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:56:08,690 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:08,690 DEBUG [M:0;jenkins-hbase4:43063] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:08,690 DEBUG [M:0;jenkins-hbase4:43063] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:56:08,690 DEBUG [M:0;jenkins-hbase4:43063] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:08,690 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.95 KB 2023-08-09 04:56:08,703 INFO [M:0;jenkins-hbase4:43063] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2a1313f7cbeb4b62a2d3c25f0f80e71e 2023-08-09 04:56:08,709 DEBUG [M:0;jenkins-hbase4:43063] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2a1313f7cbeb4b62a2d3c25f0f80e71e as hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2a1313f7cbeb4b62a2d3c25f0f80e71e 2023-08-09 04:56:08,715 INFO [M:0;jenkins-hbase4:43063] regionserver.HStore(1080): Added hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2a1313f7cbeb4b62a2d3c25f0f80e71e, entries=10, sequenceid=218, filesize=5.8 K 2023-08-09 04:56:08,716 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30423, heapSize ~36.94 KB/37824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=218, compaction requested=false 2023-08-09 04:56:08,724 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:08,724 DEBUG [M:0;jenkins-hbase4:43063] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:56:08,728 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/MasterData/WALs/jenkins-hbase4.apache.org,43063,1691556961283/jenkins-hbase4.apache.org%2C43063%2C1691556961283.jenkins-hbase4.apache.org%2C43063%2C1691556961283.regiongroup-0.1691556961422 not finished, retry = 0 2023-08-09 04:56:08,779 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:08,780 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:42717-0x101e814d6e1002b, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:08,779 INFO [RS:0;jenkins-hbase4:42717] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42717,1691556961320; zookeeper connection closed. 2023-08-09 04:56:08,781 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@48c23039] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@48c23039 2023-08-09 04:56:08,781 INFO [Listener at localhost/38001] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-09 04:56:08,829 INFO [M:0;jenkins-hbase4:43063] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-09 04:56:08,829 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:56:08,831 INFO [M:0;jenkins-hbase4:43063] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43063 2023-08-09 04:56:08,834 DEBUG [M:0;jenkins-hbase4:43063] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,43063,1691556961283 already deleted, retry=false 2023-08-09 04:56:08,935 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:08,935 INFO [M:0;jenkins-hbase4:43063] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43063,1691556961283; zookeeper connection closed. 2023-08-09 04:56:08,935 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:43063-0x101e814d6e1002a, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:08,938 INFO [Listener at localhost/38001] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:56:08,939 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:08,939 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:08,939 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:56:08,939 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:08,939 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:56:08,939 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:56:08,940 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38169 2023-08-09 04:56:08,940 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:08,942 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:08,943 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38169 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:08,946 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:381690x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:08,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38169-0x101e814d6e10032 connected 2023-08-09 04:56:08,949 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:56:08,950 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:56:08,950 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:56:08,950 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38169 2023-08-09 04:56:08,951 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38169 2023-08-09 04:56:08,951 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38169 2023-08-09 04:56:08,951 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38169 2023-08-09 04:56:08,951 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38169 2023-08-09 04:56:08,951 INFO [Listener at localhost/38001] master.HMaster(444): hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027, hbase.cluster.distributed=false 2023-08-09 04:56:08,952 DEBUG [pool-662-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-08-09 04:56:08,966 INFO [Listener at localhost/38001] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-09 04:56:08,966 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:08,967 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:08,967 INFO [Listener at localhost/38001] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-09 04:56:08,967 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-09 04:56:08,967 INFO [Listener at localhost/38001] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-09 04:56:08,967 INFO [Listener at localhost/38001] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-09 04:56:08,968 INFO [Listener at localhost/38001] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39695 2023-08-09 04:56:08,968 INFO [Listener at localhost/38001] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-09 04:56:08,971 DEBUG [Listener at localhost/38001] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-09 04:56:08,971 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:08,973 INFO [Listener at localhost/38001] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:08,974 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39695 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:08,977 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:396950x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:08,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39695-0x101e814d6e10033 connected 2023-08-09 04:56:08,980 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:56:08,981 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:56:08,982 DEBUG [Listener at localhost/38001] zookeeper.ZKUtil(164): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-09 04:56:08,986 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39695 2023-08-09 04:56:08,987 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39695 2023-08-09 04:56:08,991 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39695 2023-08-09 04:56:08,995 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39695 2023-08-09 04:56:08,998 DEBUG [Listener at localhost/38001] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39695 2023-08-09 04:56:09,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,003 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-09 04:56:09,003 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,006 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:56:09,006 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-09 04:56:09,006 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:56:09,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-09 04:56:09,010 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,38169,1691556968938 from backup master directory 2023-08-09 04:56:09,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-09 04:56:09,011 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,011 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-09 04:56:09,011 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:56:09,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,024 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-09 04:56:09,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2b4aae77 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:09,116 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7974c63d to 127.0.0.1:63198 2023-08-09 04:56:09,116 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:09,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@71d775c3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:09,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-09 04:56:09,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-09 04:56:09,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:09,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:09,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384 to hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384-dead as it is dead 2023-08-09 04:56:09,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384-dead/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 2023-08-09 04:56:09,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384-dead/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 after 0ms 2023-08-09 04:56:09,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384-dead/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 to hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 2023-08-09 04:56:09,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,42053,1691556944384-dead 2023-08-09 04:56:09,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:09,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38169%2C1691556968938.jenkins-hbase4.apache.org%2C38169%2C1691556968938.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38169,1691556968938, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/oldWALs, maxLogs=10 2023-08-09 04:56:09,144 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:56:09,145 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:56:09,145 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:56:09,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/WALs/jenkins-hbase4.apache.org,38169,1691556968938/jenkins-hbase4.apache.org%2C38169%2C1691556968938.jenkins-hbase4.apache.org%2C38169%2C1691556968938.regiongroup-0.1691556969132 2023-08-09 04:56:09,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK]] 2023-08-09 04:56:09,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:09,154 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:09,154 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:09,154 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:09,159 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:09,160 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-09 04:56:09,160 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-09 04:56:09,166 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/242b42096ae04f3190e328926c0ee389 2023-08-09 04:56:09,170 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b4e9a3dd7a57490a83c9b47baa094b5e 2023-08-09 04:56:09,170 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:09,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-09 04:56:09,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 2023-08-09 04:56:09,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 116, firstSequenceIdInLog=120, maxSequenceIdInLog=220, path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 2023-08-09 04:56:09,175 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C42053%2C1691556944384.jenkins-hbase4.apache.org%2C42053%2C1691556944384.regiongroup-0.1691556944555 2023-08-09 04:56:09,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-09 04:56:09,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/220.seqid, newMaxSeqId=220, maxSeqId=118 2023-08-09 04:56:09,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=221; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70100107, jitterRate=0.044572994112968445}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:56:09,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:56:09,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-09 04:56:09,182 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-09 04:56:09,182 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-09 04:56:09,182 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-09 04:56:09,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-09 04:56:09,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-09 04:56:09,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-09 04:56:09,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-09 04:56:09,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-09 04:56:09,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-09 04:56:09,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-09 04:56:09,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,40127,1691556935817, splitWal=true, meta=false 2023-08-09 04:56:09,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42521,1691556935911, splitWal=true, meta=false 2023-08-09 04:56:09,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33953,1691556935846, splitWal=true, meta=false 2023-08-09 04:56:09,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46479,1691556935882, splitWal=true, meta=true 2023-08-09 04:56:09,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-09 04:56:09,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-09 04:56:09,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-09 04:56:09,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,45045,1691556944435, table=hbase:meta, region=1588230740 2023-08-09 04:56:09,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-08-09 04:56:09,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45045,1691556944435 already deleted, retry=false 2023-08-09 04:56:09,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45045,1691556944435 on jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45045,1691556944435, splitWal=true, meta=true 2023-08-09 04:56:09,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,45045,1691556944435 (carryingMeta=true) jenkins-hbase4.apache.org,45045,1691556944435/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1464638e[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-09 04:56:09,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-09 04:56:09,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-09 04:56:09,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-09 04:56:09,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-09 04:56:09,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-09 04:56:09,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-09 04:56:09,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-09 04:56:09,213 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-09 04:56:09,214 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:56:09,214 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-09 04:56:09,214 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:56:09,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38169,1691556968938, sessionid=0x101e814d6e10032, setting cluster-up flag (Was=false) 2023-08-09 04:56:09,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-09 04:56:09,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-09 04:56:09,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:09,227 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/.hbase-snapshot/.tmp 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691556999238 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-09 04:56:09,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-09 04:56:09,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,239 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45045,1691556944435; numProcessing=1 2023-08-09 04:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-09 04:56:09,240 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45045,1691556944435, splitWal=true, meta=true 2023-08-09 04:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-09 04:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-09 04:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-09 04:56:09,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-09 04:56:09,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556969241,5,FailOnTimeoutGroup] 2023-08-09 04:56:09,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556969241,5,FailOnTimeoutGroup] 2023-08-09 04:56:09,241 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45045,1691556944435, splitWal=true, meta=true, isMeta: true 2023-08-09 04:56:09,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-09 04:56:09,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691556969242, completionTime=-1 2023-08-09 04:56:09,242 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-09 04:56:09,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-09 04:56:09,242 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435-splitting 2023-08-09 04:56:09,243 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435-splitting dir is empty, no logs to split. 2023-08-09 04:56:09,243 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,45045,1691556944435 WAL count=0, meta=true 2023-08-09 04:56:09,245 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435-splitting dir is empty, no logs to split. 2023-08-09 04:56:09,245 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,45045,1691556944435 WAL count=0, meta=true 2023-08-09 04:56:09,245 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45045,1691556944435 WAL splitting is done? wals=0, meta=true 2023-08-09 04:56:09,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-09 04:56:09,247 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-09 04:56:09,248 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:09,300 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(951): ClusterId : 4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:56:09,300 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-09 04:56:09,302 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-09 04:56:09,302 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-09 04:56:09,304 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-09 04:56:09,306 DEBUG [RS:0;jenkins-hbase4:39695] zookeeper.ReadOnlyZKClient(139): Connect 0x1536998d to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:09,310 DEBUG [RS:0;jenkins-hbase4:39695] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bf1655b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:09,310 DEBUG [RS:0;jenkins-hbase4:39695] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fcef2fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:56:09,318 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39695 2023-08-09 04:56:09,319 INFO [RS:0;jenkins-hbase4:39695] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-09 04:56:09,319 INFO [RS:0;jenkins-hbase4:39695] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-09 04:56:09,319 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1022): About to register with Master. 2023-08-09 04:56:09,319 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38169,1691556968938 with isa=jenkins-hbase4.apache.org/172.31.14.131:39695, startcode=1691556968966 2023-08-09 04:56:09,319 DEBUG [RS:0;jenkins-hbase4:39695] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-09 04:56:09,321 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33777, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-08-09 04:56:09,322 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38169] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,323 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027 2023-08-09 04:56:09,323 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39381 2023-08-09 04:56:09,323 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-09 04:56:09,324 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:09,324 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10026, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:09,324 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:56:09,325 DEBUG [RS:0;jenkins-hbase4:39695] zookeeper.ZKUtil(162): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,325 WARN [RS:0;jenkins-hbase4:39695] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-09 04:56:09,325 INFO [RS:0;jenkins-hbase4:39695] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:09,325 INFO [RS:0;jenkins-hbase4:39695] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:09,325 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,327 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39695,1691556968966] 2023-08-09 04:56:09,330 DEBUG [RS:0;jenkins-hbase4:39695] zookeeper.ZKUtil(162): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,331 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:56:09,331 INFO [RS:0;jenkins-hbase4:39695] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-09 04:56:09,333 INFO [RS:0;jenkins-hbase4:39695] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-09 04:56:09,333 INFO [RS:0;jenkins-hbase4:39695] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-09 04:56:09,333 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,333 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-09 04:56:09,335 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,335 DEBUG [RS:0;jenkins-hbase4:39695] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-09 04:56:09,338 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,338 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,338 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-09 04:56:09,349 INFO [RS:0;jenkins-hbase4:39695] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-09 04:56:09,349 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39695,1691556968966-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:09,359 INFO [RS:0;jenkins-hbase4:39695] regionserver.Replication(203): jenkins-hbase4.apache.org,39695,1691556968966 started 2023-08-09 04:56:09,359 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39695,1691556968966, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39695, sessionid=0x101e814d6e10033 2023-08-09 04:56:09,359 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-09 04:56:09,359 DEBUG [RS:0;jenkins-hbase4:39695] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,359 DEBUG [RS:0;jenkins-hbase4:39695] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39695,1691556968966' 2023-08-09 04:56:09,359 DEBUG [RS:0;jenkins-hbase4:39695] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-09 04:56:09,360 DEBUG [RS:0;jenkins-hbase4:39695] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-09 04:56:09,360 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-09 04:56:09,360 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-09 04:56:09,360 DEBUG [RS:0;jenkins-hbase4:39695] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,360 DEBUG [RS:0;jenkins-hbase4:39695] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39695,1691556968966' 2023-08-09 04:56:09,360 DEBUG [RS:0;jenkins-hbase4:39695] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-09 04:56:09,361 DEBUG [RS:0;jenkins-hbase4:39695] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-09 04:56:09,361 DEBUG [RS:0;jenkins-hbase4:39695] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-09 04:56:09,361 INFO [RS:0;jenkins-hbase4:39695] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-09 04:56:09,361 INFO [RS:0;jenkins-hbase4:39695] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-09 04:56:09,398 DEBUG [jenkins-hbase4:38169] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-09 04:56:09,400 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39695,1691556968966, state=OPENING 2023-08-09 04:56:09,404 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-09 04:56:09,404 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-09 04:56:09,404 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39695,1691556968966}] 2023-08-09 04:56:09,461 INFO [RS:0;jenkins-hbase4:39695] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:09,463 INFO [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39695%2C1691556968966.jenkins-hbase4.apache.org%2C39695%2C1691556968966.regiongroup-0, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:56:09,479 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:56:09,479 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:56:09,479 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:56:09,481 INFO [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966/jenkins-hbase4.apache.org%2C39695%2C1691556968966.jenkins-hbase4.apache.org%2C39695%2C1691556968966.regiongroup-0.1691556969464 2023-08-09 04:56:09,481 DEBUG [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK]] 2023-08-09 04:56:09,561 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:09,561 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:56:09,562 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38024, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:56:09,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-09 04:56:09,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:09,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:09,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:09,568 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39695%2C1691556968966.meta, suffix=.meta, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:56:09,582 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:56:09,582 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:56:09,582 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK] 2023-08-09 04:56:09,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966/jenkins-hbase4.apache.org%2C39695%2C1691556968966.meta.1691556969568.meta 2023-08-09 04:56:09,585 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK], DatanodeInfoWithStorage[127.0.0.1:37263,DS-4ff5e5dd-92cd-4741-b3cb-ee483e2c1835,DISK], DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK]] 2023-08-09 04:56:09,585 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:09,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-09 04:56:09,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-09 04:56:09,586 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-09 04:56:09,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-09 04:56:09,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:09,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-09 04:56:09,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-09 04:56:09,587 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-09 04:56:09,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:56:09,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info 2023-08-09 04:56:09,589 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-09 04:56:09,599 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/6bbf0f789adf465aa07c7a710d6dbf92 2023-08-09 04:56:09,603 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/e298eec93fb24463bb4dc7984a19b330 2023-08-09 04:56:09,604 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:09,604 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-09 04:56:09,605 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:56:09,605 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/rep_barrier 2023-08-09 04:56:09,605 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-09 04:56:09,606 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:09,606 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-09 04:56:09,607 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:56:09,607 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table 2023-08-09 04:56:09,607 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-09 04:56:09,612 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/table/f077c49e23464cb69defed46ad23be28 2023-08-09 04:56:09,613 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:09,613 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:56:09,614 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740 2023-08-09 04:56:09,617 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-09 04:56:09,618 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-09 04:56:09,619 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63886188, jitterRate=-0.04802161455154419}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-09 04:56:09,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-09 04:56:09,620 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1691556969560 2023-08-09 04:56:09,624 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-09 04:56:09,625 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-09 04:56:09,625 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39695,1691556968966, state=OPEN 2023-08-09 04:56:09,627 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-09 04:56:09,628 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-09 04:56:09,629 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-08-09 04:56:09,630 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39695,1691556968966 in 224 msec 2023-08-09 04:56:09,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-08-09 04:56:09,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 384 msec 2023-08-09 04:56:10,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1602ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-08-09 04:56:11,745 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:56:12,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-08-09 04:56:13,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-09 04:56:13,749 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-09 04:56:13,753 DEBUG [hconnection-0x36f15e41-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:13,755 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49728, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:56:13,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=36954931527a871a4ac33b21c04863f3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45045,1691556944435, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435, openSeqNum=10 2023-08-09 04:56:13,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45045,1691556944435, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435, openSeqNum=11 2023-08-09 04:56:13,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=629be257c9d8450abcf2763c66f8a37a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45045,1691556944435, regionLocation=jenkins-hbase4.apache.org,45045,1691556944435, openSeqNum=12 2023-08-09 04:56:13,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-09 04:56:13,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691557033769 2023-08-09 04:56:13,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691557093769 2023-08-09 04:56:13,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 19 msec 2023-08-09 04:56:13,791 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45045,1691556944435 had 4 regions 2023-08-09 04:56:13,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38169,1691556968938-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:13,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38169,1691556968938-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:13,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38169,1691556968938-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:13,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38169, period=300000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:13,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-09 04:56:13,791 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. is NOT online; state={36954931527a871a4ac33b21c04863f3 state=OPEN, ts=1691556973768, server=jenkins-hbase4.apache.org,45045,1691556944435}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-09 04:56:13,797 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45045,1691556944435, splitWal=true, meta=true, isMeta: false 2023-08-09 04:56:13,801 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,45045,1691556944435/hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., unknown_server=jenkins-hbase4.apache.org,45045,1691556944435/t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., unknown_server=jenkins-hbase4.apache.org,45045,1691556944435/t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:13,801 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435-splitting dir is empty, no logs to split. 2023-08-09 04:56:13,801 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,45045,1691556944435 WAL count=0, meta=false 2023-08-09 04:56:13,803 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,45045,1691556944435-splitting dir is empty, no logs to split. 2023-08-09 04:56:13,803 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,45045,1691556944435 WAL count=0, meta=false 2023-08-09 04:56:13,803 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45045,1691556944435 WAL splitting is done? wals=0, meta=false 2023-08-09 04:56:13,807 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN}] 2023-08-09 04:56:13,808 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN 2023-08-09 04:56:13,809 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN 2023-08-09 04:56:13,809 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN 2023-08-09 04:56:13,809 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:13,809 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:13,810 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-09 04:56:13,960 DEBUG [jenkins-hbase4:38169] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-09 04:56:13,964 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=36954931527a871a4ac33b21c04863f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:13,964 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=629be257c9d8450abcf2763c66f8a37a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:13,964 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:13,964 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556973964"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556973964"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556973964"}]},"ts":"1691556973964"} 2023-08-09 04:56:13,964 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556973963"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556973963"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556973963"}]},"ts":"1691556973963"} 2023-08-09 04:56:13,964 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556973964"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691556973964"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691556973964"}]},"ts":"1691556973964"} 2023-08-09 04:56:13,967 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 36954931527a871a4ac33b21c04863f3, server=jenkins-hbase4.apache.org,39695,1691556968966}] 2023-08-09 04:56:13,967 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=30, state=RUNNABLE; OpenRegionProcedure 629be257c9d8450abcf2763c66f8a37a, server=jenkins-hbase4.apache.org,39695,1691556968966}] 2023-08-09 04:56:13,970 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=31, state=RUNNABLE; OpenRegionProcedure baf34b839cf96847099b70ccf7c3d7ea, server=jenkins-hbase4.apache.org,39695,1691556968966}] 2023-08-09 04:56:14,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:14,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-09 04:56:14,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39695%2C1691556968966.jenkins-hbase4.apache.org%2C39695%2C1691556968966.regiongroup-1, suffix=, logDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966, archiveDir=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs, maxLogs=10 2023-08-09 04:56:14,133 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:14,133 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36954931527a871a4ac33b21c04863f3, NAME => 'hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:14,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:14,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,134 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,135 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,137 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info 2023-08-09 04:56:14,137 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info 2023-08-09 04:56:14,137 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36954931527a871a4ac33b21c04863f3 columnFamilyName info 2023-08-09 04:56:14,142 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK] 2023-08-09 04:56:14,143 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK] 2023-08-09 04:56:14,144 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK] 2023-08-09 04:56:14,145 DEBUG [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/info/33bb63d4ef714b58996b9beecc59a90a 2023-08-09 04:56:14,151 INFO [StoreOpener-36954931527a871a4ac33b21c04863f3-1] regionserver.HStore(310): Store=36954931527a871a4ac33b21c04863f3/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:14,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/WALs/jenkins-hbase4.apache.org,39695,1691556968966/jenkins-hbase4.apache.org%2C39695%2C1691556968966.jenkins-hbase4.apache.org%2C39695%2C1691556968966.regiongroup-1.1691556974128 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-1ac82927-5264-4da4-9391-616e449c5539,DISK], DatanodeInfoWithStorage[127.0.0.1:40039,DS-5a15e612-c384-4cea-a2d1-5c7c9a48e617,DISK], DatanodeInfoWithStorage[127.0.0.1:38281,DS-7fb9345f-d2ec-41a1-90f6-a003a34848bf,DISK]] 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => baf34b839cf96847099b70ccf7c3d7ea, NAME => 't1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,155 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,156 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,156 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1 2023-08-09 04:56:14,156 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1 2023-08-09 04:56:14,157 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baf34b839cf96847099b70ccf7c3d7ea columnFamilyName cf1 2023-08-09 04:56:14,160 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:14,161 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36954931527a871a4ac33b21c04863f3; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64145264, jitterRate=-0.044161081314086914}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-09 04:56:14,161 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:56:14,162 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., pid=33, masterSystemTime=1691556974121 2023-08-09 04:56:14,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:14,165 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:14,165 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=36954931527a871a4ac33b21c04863f3, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:14,165 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05caab739a2340a0b24d14c800479d1a 2023-08-09 04:56:14,166 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/05caab739a2340a0b24d14c800479d1a 2023-08-09 04:56:14,166 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691556974165"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556974165"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556974165"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556974165"}]},"ts":"1691556974165"} 2023-08-09 04:56:14,168 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-08-09 04:56:14,169 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 36954931527a871a4ac33b21c04863f3, server=jenkins-hbase4.apache.org,39695,1691556968966 in 201 msec 2023-08-09 04:56:14,170 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=36954931527a871a4ac33b21c04863f3, ASSIGN in 364 msec 2023-08-09 04:56:14,173 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/98fba2c9188748208109db5c299fb897 2023-08-09 04:56:14,173 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(310): Store=baf34b839cf96847099b70ccf7c3d7ea/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:14,173 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,174 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/norep 2023-08-09 04:56:14,174 DEBUG [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/norep 2023-08-09 04:56:14,174 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baf34b839cf96847099b70ccf7c3d7ea columnFamilyName norep 2023-08-09 04:56:14,175 INFO [StoreOpener-baf34b839cf96847099b70ccf7c3d7ea-1] regionserver.HStore(310): Store=baf34b839cf96847099b70ccf7c3d7ea/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:14,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:56:14,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:14,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened baf34b839cf96847099b70ccf7c3d7ea; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75482811, jitterRate=0.124781534075737}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:56:14,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:56:14,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., pid=35, masterSystemTime=1691556974121 2023-08-09 04:56:14,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:14,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:14,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:14,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 629be257c9d8450abcf2763c66f8a37a, NAME => 't2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.', STARTKEY => '', ENDKEY => ''} 2023-08-09 04:56:14,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,186 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=baf34b839cf96847099b70ccf7c3d7ea, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:14,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-09 04:56:14,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,186 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556974186"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556974186"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556974186"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556974186"}]},"ts":"1691556974186"} 2023-08-09 04:56:14,189 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=31 2023-08-09 04:56:14,189 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=31, state=SUCCESS; OpenRegionProcedure baf34b839cf96847099b70ccf7c3d7ea, server=jenkins-hbase4.apache.org,39695,1691556968966 in 219 msec 2023-08-09 04:56:14,191 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=baf34b839cf96847099b70ccf7c3d7ea, ASSIGN in 384 msec 2023-08-09 04:56:14,192 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1 2023-08-09 04:56:14,192 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1 2023-08-09 04:56:14,192 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 629be257c9d8450abcf2763c66f8a37a columnFamilyName cf1 2023-08-09 04:56:14,200 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/1b482211d41c43d88278a7605ec6ae58 2023-08-09 04:56:14,206 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 625d721f48ff4ff6ad709109dff79fcb 2023-08-09 04:56:14,206 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(539): loaded hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/625d721f48ff4ff6ad709109dff79fcb 2023-08-09 04:56:14,206 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(310): Store=629be257c9d8450abcf2763c66f8a37a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:14,206 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,207 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/norep 2023-08-09 04:56:14,207 DEBUG [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/norep 2023-08-09 04:56:14,208 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 629be257c9d8450abcf2763c66f8a37a columnFamilyName norep 2023-08-09 04:56:14,208 INFO [StoreOpener-629be257c9d8450abcf2763c66f8a37a-1] regionserver.HStore(310): Store=629be257c9d8450abcf2763c66f8a37a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-09 04:56:14,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-09 04:56:14,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:14,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 629be257c9d8450abcf2763c66f8a37a; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64676879, jitterRate=-0.03623940050601959}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-09 04:56:14,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:56:14,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a., pid=34, masterSystemTime=1691556974121 2023-08-09 04:56:14,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:14,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:14,217 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=629be257c9d8450abcf2763c66f8a37a, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:14,217 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691556974217"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691556974217"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691556974217"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691556974217"}]},"ts":"1691556974217"} 2023-08-09 04:56:14,220 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=30 2023-08-09 04:56:14,220 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=30, state=SUCCESS; OpenRegionProcedure 629be257c9d8450abcf2763c66f8a37a, server=jenkins-hbase4.apache.org,39695,1691556968966 in 252 msec 2023-08-09 04:56:14,222 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-08-09 04:56:14,223 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45045,1691556944435 after splitting done 2023-08-09 04:56:14,223 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=629be257c9d8450abcf2763c66f8a37a, ASSIGN in 415 msec 2023-08-09 04:56:14,223 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45045,1691556944435 from processing; numProcessing=0 2023-08-09 04:56:14,224 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45045,1691556944435, splitWal=true, meta=true in 5.0240 sec 2023-08-09 04:56:14,359 INFO [jenkins-hbase4:39695Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:56:14,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-08-09 04:56:14,809 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-09 04:56:14,813 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-09 04:56:14,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.802sec 2023-08-09 04:56:14,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-09 04:56:14,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-09 04:56:14,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-09 04:56:14,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38169,1691556968938-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-09 04:56:14,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38169,1691556968938-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-09 04:56:14,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-09 04:56:14,905 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x0eef6066 to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:14,909 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ca47bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:14,910 DEBUG [hconnection-0x7b202aaf-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:14,912 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49730, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:56:14,914 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-09 04:56:14,914 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0eef6066 to 127.0.0.1:63198 2023-08-09 04:56:14,914 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:14,915 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:38169 after: jenkins-hbase4.apache.org:38169 2023-08-09 04:56:14,915 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(139): Connect 0x57b3bf2c to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:14,919 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d3c8227, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:15,332 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-09 04:56:15,332 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-09 04:56:15,333 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-09 04:56:15,333 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-09 04:56:15,421 DEBUG [hconnection-0x27142824-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:15,423 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49738, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-09 04:56:15,453 INFO [Listener at localhost/38001] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1691556975453 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:15,456 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication16915569754530x0, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:15,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1691556975453-0x101e814d6e10038 connected Start Replication Server start 2023-08-09 04:56:15,460 INFO [Listener at localhost/38001] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-09 04:56:15,460 INFO [Listener at localhost/38001] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-09 04:56:15,473 INFO [Listener at localhost/38001] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-09 04:56:15,475 DEBUG [Listener at localhost/38001] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-09 04:56:15,488 INFO [Listener at localhost/38001] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:56:15,490 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,42717,1691556961320] other RSs: [] 2023-08-09 04:56:15,516 DEBUG [Listener at localhost/38001.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x67208aff to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:15,521 DEBUG [Listener at localhost/38001.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5c368270, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:15,521 INFO [Listener at localhost/38001.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:15,523 DEBUG [Listener at localhost/38001.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:15,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e1003a connected 2023-08-09 04:56:15,524 INFO [Listener at localhost/38001.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:56:18,574 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42717,1691556961320/1's WALs to 1691556975460.syncuptool.replication.org,1234,1 2023-08-09 04:56:18,576 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 with data PBUF\x08\xD8\x05 2023-08-09 04:56:18,576 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240 with data PBUF\x08\xB7\x06 2023-08-09 04:56:18,578 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,42717,1691556961320/1's WALs to 1691556975460.syncuptool.replication.org,1234,1 2023-08-09 04:56:19,359 INFO [jenkins-hbase4:39695Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:56:19,637 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:56:20,579 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:20,580 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:20,590 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:20,590 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,42717,1691556961320, ReplicationSource: 1, currentBandwidth=0 2023-08-09 04:56:20,591 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,610 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] zookeeper.ReadOnlyZKClient(139): Connect 0x41562bcd to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:20,616 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@255f4b5e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:20,616 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63198 2023-08-09 04:56:20,618 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-09 04:56:20,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101e814d6e1003c connected 2023-08-09 04:56:20,620 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,42717,1691556961320 (queues=2) is replicating from cluster=2cfbe88b-b307-4b47-a587-5b8f3b8f6da5 to cluster=4247f419-964c-40f1-a0c4-21791f0035af 2023-08-09 04:56:20,620 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0 2023-08-09 04:56:20,623 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42717,1691556961320, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:56:20,627 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1 2023-08-09 04:56:20,627 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0 2023-08-09 04:56:20,628 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42717,1691556961320, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-09 04:56:20,630 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1 2023-08-09 04:56:20,645 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 2023-08-09 04:56:20,645 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-09 04:56:20,645 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42717,1691556961320 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:56:20,646 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,651 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240 2023-08-09 04:56:20,651 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-08-09 04:56:20,651 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-09 04:56:20,653 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42717,1691556961320 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/4d1b412f72764eba378319d319bc7848/112=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/113=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/114=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/115=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/116=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/117=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/118=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/119=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/120=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/121=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/122=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/123=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/124=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/125=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/126=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/127=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/128=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/129=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/130=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/131=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/132=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/133=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/134=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/135=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/136=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/137=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/138=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/139=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/140=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/141=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/142=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/143=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/144=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/145=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/146=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/147=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/148=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/149=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/150=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/151=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/152=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/153=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/154=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/155=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/156=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/157=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/158=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/159=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/160=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/161=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/162=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/163=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/164=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/165=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/166=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/167=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/168=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/169=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/170=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/171=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/172=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/173=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/174=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/175=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/176=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/177=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/178=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/179=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/180=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/181=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/182=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/183=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/184=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/185=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/186=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/187=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/188=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/189=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/190=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/191=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/192=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/193=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/194=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/195=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/196=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/197=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/198=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/199=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/200=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/201=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/202=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/203=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/204=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/205=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/206=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/207=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/208=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/209=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/210=[#edits: 1 = ],208}, {t1_syncup/4d1b412f72764eba378319d319bc7848/211=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/212=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/213=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/214=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/215=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/216=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/217=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/218=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/219=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/220=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/221=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/222=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/223=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/224=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/225=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/226=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/227=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/228=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/229=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/230=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/231=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/232=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/233=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/234=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/235=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/236=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/237=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/238=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/239=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/240=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/241=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/242=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/243=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/244=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/245=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/246=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/247=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/248=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/249=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/250=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/251=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/252=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/253=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/254=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/255=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/256=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/257=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/258=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/259=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/260=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/261=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/262=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/263=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/264=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/265=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/266=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/267=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/268=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/269=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/270=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/271=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/272=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/273=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/274=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/275=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/276=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/277=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/278=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/279=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/280=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/281=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/282=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/283=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/284=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/285=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/286=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/287=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/288=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/289=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/290=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/291=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/292=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/293=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/294=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/295=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/296=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/297=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/298=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/299=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/300=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/301=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/302=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/303=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/304=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/305=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/306=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/307=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/308=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/309=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/310=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/311=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/312=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/313=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/314=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/315=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/316=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/317=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/318=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/319=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/320=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/321=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/322=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/323=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/324=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/325=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/326=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/327=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/328=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/329=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/330=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/331=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/332=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/333=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/334=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/335=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/336=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/337=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/338=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/339=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/340=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/341=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/342=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/343=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/344=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/345=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/346=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/347=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/348=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/349=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/350=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/351=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/352=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/353=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/354=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/355=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/356=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/357=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/358=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/359=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/360=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/361=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/362=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/363=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/364=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/365=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/366=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/367=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/368=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/369=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/370=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/371=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/372=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/373=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/374=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/375=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/376=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/377=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/378=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/379=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/380=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/381=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/382=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/383=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/384=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/385=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/386=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/387=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/388=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/389=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/390=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/391=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/392=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/393=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/394=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/395=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/396=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/397=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/398=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/399=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/400=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/401=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/402=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/403=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/404=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/405=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/406=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/407=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/408=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/409=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/410=[#edits: 1 = ],208}, {t2_syncup/2ccff9a5354735fa815894d8a555d598/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:45825/user/jenkins/test-data/78b42d81-19e5-988d-5e41-fa673d6e59d1/oldWALs/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240, lastWalPosition=34707, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-08-09 04:56:20,660 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-09 04:56:20,664 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34746, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-09 04:56:20,669 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39695] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-09 04:56:20,669 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39695] zookeeper.ReadOnlyZKClient(139): Connect 0x323b2a4f to 127.0.0.1:63198 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-09 04:56:20,676 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39695] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3cfcd278, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-09 04:56:20,679 DEBUG [hconnection-0x38c3413f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-09 04:56:20,680 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34752, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-08-09 04:56:20,705 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39695] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-09 04:56:20,706 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42717,1691556961320] 2023-08-09 04:56:20,708 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1.1691556966240] 2023-08-09 04:56:20,711 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42717,1691556961320 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-09 04:56:20,711 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,747 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,747 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] zookeeper.ZKUtil(162): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1691556975460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 2023-08-09 04:56:20,752 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691556975460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42717,1691556961320/jenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0.1691556961727 2023-08-09 04:56:20,752 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1691556975460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,752 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691556975460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,752 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-0,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42717,1691556961320 with the following stats: Total replicated edits: 300, current progress: 2023-08-09 04:56:20,852 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42717,1691556961320 2023-08-09 04:56:20,853 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] zookeeper.ZKUtil(243): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Unable to list children of znode /1/replication/rs/1691556975460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42717,1691556961320 because node does not exist (not an error) 2023-08-09 04:56:20,853 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320.replicationSource.shipperjenkins-hbase4.apache.org%2C42717%2C1691556961320.jenkins-hbase4.apache.org%2C42717%2C1691556961320.regiongroup-1,1-jenkins-hbase4.apache.org,42717,1691556961320] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42717,1691556961320 with the following stats: Total replicated edits: 300, current progress: 2023-08-09 04:56:24,359 INFO [jenkins-hbase4:39695Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-09 04:56:24,359 INFO [jenkins-hbase4:39695Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-08-09 04:56:25,490 INFO [Listener at localhost/38001] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-09 04:56:25,594 DEBUG [Listener at localhost/38001.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e1003a, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:25,594 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67208aff to 127.0.0.1:63198 2023-08-09 04:56:25,595 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:25,797 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:25,797 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691556975453-0x101e814d6e10038, quorum=127.0.0.1:63198, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:25,805 INFO [Listener at localhost/38001] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-08-09 04:56:25,805 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-09 04:56:25,805 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57b3bf2c to 127.0.0.1:63198 2023-08-09 04:56:25,805 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:25,806 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-09 04:56:25,806 DEBUG [Listener at localhost/38001] util.JVMClusterUtil(257): Found active master hash=1423243925, stopped=false 2023-08-09 04:56:25,806 INFO [Listener at localhost/38001] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:25,809 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:56:25,809 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-09 04:56:25,809 INFO [Listener at localhost/38001] procedure2.ProcedureExecutor(629): Stopping 2023-08-09 04:56:25,809 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:56:25,811 DEBUG [Listener at localhost/38001] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2b4aae77 to 127.0.0.1:63198 2023-08-09 04:56:25,811 DEBUG [Listener at localhost/38001] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:25,811 INFO [Listener at localhost/38001] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39695,1691556968966' ***** 2023-08-09 04:56:25,811 INFO [Listener at localhost/38001] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-09 04:56:25,811 INFO [RS:0;jenkins-hbase4:39695] regionserver.HeapMemoryManager(220): Stopping 2023-08-09 04:56:25,812 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-09 04:56:25,812 INFO [RS:0;jenkins-hbase4:39695] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-09 04:56:25,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:56:25,812 INFO [RS:0;jenkins-hbase4:39695] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-09 04:56:25,813 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(3305): Received CLOSE for 36954931527a871a4ac33b21c04863f3 2023-08-09 04:56:25,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-09 04:56:25,813 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(3305): Received CLOSE for baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:25,813 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(3305): Received CLOSE for 629be257c9d8450abcf2763c66f8a37a 2023-08-09 04:56:25,814 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:25,814 DEBUG [RS:0;jenkins-hbase4:39695] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1536998d to 127.0.0.1:63198 2023-08-09 04:56:25,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36954931527a871a4ac33b21c04863f3, disabling compactions & flushes 2023-08-09 04:56:25,815 DEBUG [RS:0;jenkins-hbase4:39695] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:25,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:25,815 INFO [RS:0;jenkins-hbase4:39695] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-09 04:56:25,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:25,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. after waiting 0 ms 2023-08-09 04:56:25,815 INFO [RS:0;jenkins-hbase4:39695] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-09 04:56:25,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:25,815 INFO [RS:0;jenkins-hbase4:39695] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-09 04:56:25,815 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-09 04:56:25,818 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-09 04:56:25,818 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1478): Online Regions={36954931527a871a4ac33b21c04863f3=hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3., baf34b839cf96847099b70ccf7c3d7ea=t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea., 1588230740=hbase:meta,,1.1588230740, 629be257c9d8450abcf2763c66f8a37a=t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a.} 2023-08-09 04:56:25,820 DEBUG [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1504): Waiting on 1588230740, 36954931527a871a4ac33b21c04863f3, 629be257c9d8450abcf2763c66f8a37a, baf34b839cf96847099b70ccf7c3d7ea 2023-08-09 04:56:25,821 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-09 04:56:25,821 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-09 04:56:25,821 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-09 04:56:25,821 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-09 04:56:25,821 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-09 04:56:25,821 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-08-09 04:56:25,831 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/namespace/36954931527a871a4ac33b21c04863f3/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-08-09 04:56:25,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36954931527a871a4ac33b21c04863f3: 2023-08-09 04:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691556936956.36954931527a871a4ac33b21c04863f3. 2023-08-09 04:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing baf34b839cf96847099b70ccf7c3d7ea, disabling compactions & flushes 2023-08-09 04:56:25,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:25,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:25,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. after waiting 0 ms 2023-08-09 04:56:25,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:25,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing baf34b839cf96847099b70ccf7c3d7ea 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-08-09 04:56:25,851 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/info/1c7f5f740ebd4146ba65a6d4ee8a9328 2023-08-09 04:56:25,853 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:56:25,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/.tmp/cf1/94298cedb7fe4ef5b7df74b212b6a8b6 2023-08-09 04:56:25,860 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/.tmp/info/1c7f5f740ebd4146ba65a6d4ee8a9328 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/1c7f5f740ebd4146ba65a6d4ee8a9328 2023-08-09 04:56:25,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/.tmp/cf1/94298cedb7fe4ef5b7df74b212b6a8b6 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/94298cedb7fe4ef5b7df74b212b6a8b6 2023-08-09 04:56:25,865 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/info/1c7f5f740ebd4146ba65a6d4ee8a9328, entries=24, sequenceid=43, filesize=7.4 K 2023-08-09 04:56:25,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/cf1/94298cedb7fe4ef5b7df74b212b6a8b6, entries=100, sequenceid=21, filesize=8.3 K 2023-08-09 04:56:25,866 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 45ms, sequenceid=43, compaction requested=true 2023-08-09 04:56:25,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for baf34b839cf96847099b70ccf7c3d7ea in 37ms, sequenceid=21, compaction requested=true 2023-08-09 04:56:25,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t1_syncup/baf34b839cf96847099b70ccf7c3d7ea/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-08-09 04:56:25,888 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-08-09 04:56:25,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:25,889 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-09 04:56:25,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for baf34b839cf96847099b70ccf7c3d7ea: 2023-08-09 04:56:25,890 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-09 04:56:25,890 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-09 04:56:25,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691556939023.baf34b839cf96847099b70ccf7c3d7ea. 2023-08-09 04:56:25,890 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-09 04:56:25,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 629be257c9d8450abcf2763c66f8a37a, disabling compactions & flushes 2023-08-09 04:56:25,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:25,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:25,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. after waiting 0 ms 2023-08-09 04:56:25,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:25,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 629be257c9d8450abcf2763c66f8a37a 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-08-09 04:56:25,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/.tmp/cf1/fa1635d9f86647e8a6a75446913ea6c5 2023-08-09 04:56:25,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/.tmp/cf1/fa1635d9f86647e8a6a75446913ea6c5 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/fa1635d9f86647e8a6a75446913ea6c5 2023-08-09 04:56:25,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/cf1/fa1635d9f86647e8a6a75446913ea6c5, entries=200, sequenceid=22, filesize=12.2 K 2023-08-09 04:56:25,923 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 629be257c9d8450abcf2763c66f8a37a in 32ms, sequenceid=22, compaction requested=true 2023-08-09 04:56:25,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/data/default/t2_syncup/629be257c9d8450abcf2763c66f8a37a/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-08-09 04:56:25,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 629be257c9d8450abcf2763c66f8a37a: 2023-08-09 04:56:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691556939638.629be257c9d8450abcf2763c66f8a37a. 2023-08-09 04:56:26,021 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39695,1691556968966; all regions closed. 2023-08-09 04:56:26,026 DEBUG [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:56:26,027 INFO [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39695%2C1691556968966.meta:.meta(num 1691556969568) 2023-08-09 04:56:26,034 DEBUG [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:56:26,034 INFO [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39695%2C1691556968966.jenkins-hbase4.apache.org%2C39695%2C1691556968966.regiongroup-1:(num 1691556974128) 2023-08-09 04:56:26,039 DEBUG [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/oldWALs 2023-08-09 04:56:26,039 INFO [RS:0;jenkins-hbase4:39695] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39695%2C1691556968966.jenkins-hbase4.apache.org%2C39695%2C1691556968966.regiongroup-0:(num 1691556969464) 2023-08-09 04:56:26,039 DEBUG [RS:0;jenkins-hbase4:39695] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:26,039 INFO [RS:0;jenkins-hbase4:39695] regionserver.LeaseManager(133): Closed leases 2023-08-09 04:56:26,040 INFO [RS:0;jenkins-hbase4:39695] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-09 04:56:26,040 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:56:26,040 DEBUG [RS:0;jenkins-hbase4:39695] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x323b2a4f to 127.0.0.1:63198 2023-08-09 04:56:26,040 DEBUG [RS:0;jenkins-hbase4:39695] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:26,041 INFO [RS:0;jenkins-hbase4:39695] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39695 2023-08-09 04:56:26,044 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e10026, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:26,044 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,39695,1691556968966 2023-08-09 04:56:26,044 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101e814d6e1003c, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:26,044 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:26,044 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:56:26,044 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-09 04:56:26,044 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-09 04:56:26,045 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39695,1691556968966] 2023-08-09 04:56:26,045 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39695,1691556968966; numProcessing=1 2023-08-09 04:56:26,046 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,39695,1691556968966 already deleted, retry=false 2023-08-09 04:56:26,046 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39695,1691556968966 expired; onlineServers=0 2023-08-09 04:56:26,046 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38169,1691556968938' ***** 2023-08-09 04:56:26,046 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-09 04:56:26,047 DEBUG [M:0;jenkins-hbase4:38169] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50eb64d7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-09 04:56:26,047 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38169,1691556968938 2023-08-09 04:56:26,047 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38169,1691556968938; all regions closed. 2023-08-09 04:56:26,047 DEBUG [M:0;jenkins-hbase4:38169] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-09 04:56:26,047 DEBUG [M:0;jenkins-hbase4:38169] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-09 04:56:26,047 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-09 04:56:26,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556969241] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691556969241,5,FailOnTimeoutGroup] 2023-08-09 04:56:26,047 DEBUG [M:0;jenkins-hbase4:38169] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-09 04:56:26,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556969241] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691556969241,5,FailOnTimeoutGroup] 2023-08-09 04:56:26,048 INFO [M:0;jenkins-hbase4:38169] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-09 04:56:26,049 INFO [M:0;jenkins-hbase4:38169] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-09 04:56:26,049 INFO [M:0;jenkins-hbase4:38169] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-09 04:56:26,049 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-09 04:56:26,049 DEBUG [M:0;jenkins-hbase4:38169] master.HMaster(1512): Stopping service threads 2023-08-09 04:56:26,049 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-09 04:56:26,049 INFO [M:0;jenkins-hbase4:38169] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-09 04:56:26,050 ERROR [M:0;jenkins-hbase4:38169] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-09 04:56:26,050 INFO [M:0;jenkins-hbase4:38169] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-09 04:56:26,050 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-09 04:56:26,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-09 04:56:26,050 DEBUG [M:0;jenkins-hbase4:38169] zookeeper.ZKUtil(398): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-09 04:56:26,050 WARN [M:0;jenkins-hbase4:38169] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-09 04:56:26,050 INFO [M:0;jenkins-hbase4:38169] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-09 04:56:26,051 INFO [M:0;jenkins-hbase4:38169] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-09 04:56:26,051 DEBUG [M:0;jenkins-hbase4:38169] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-09 04:56:26,051 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:26,051 DEBUG [M:0;jenkins-hbase4:38169] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:26,051 DEBUG [M:0;jenkins-hbase4:38169] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-09 04:56:26,051 DEBUG [M:0;jenkins-hbase4:38169] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:26,051 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-08-09 04:56:26,064 INFO [M:0;jenkins-hbase4:38169] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=289 (bloomFilter=true), to=hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/578f839ed770469aa7222e99a1e25ef4 2023-08-09 04:56:26,069 DEBUG [M:0;jenkins-hbase4:38169] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/578f839ed770469aa7222e99a1e25ef4 as hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/578f839ed770469aa7222e99a1e25ef4 2023-08-09 04:56:26,074 INFO [M:0;jenkins-hbase4:38169] regionserver.HStore(1080): Added hdfs://localhost:39381/user/jenkins/test-data/a13e912d-3d87-0676-c767-c147deefe027/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/578f839ed770469aa7222e99a1e25ef4, entries=9, sequenceid=289, filesize=5.5 K 2023-08-09 04:56:26,075 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=289, compaction requested=true 2023-08-09 04:56:26,075 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-09 04:56:26,075 DEBUG [M:0;jenkins-hbase4:38169] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-09 04:56:26,079 INFO [M:0;jenkins-hbase4:38169] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-09 04:56:26,079 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-09 04:56:26,080 INFO [M:0;jenkins-hbase4:38169] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38169 2023-08-09 04:56:26,083 DEBUG [M:0;jenkins-hbase4:38169] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,38169,1691556968938 already deleted, retry=false 2023-08-09 04:56:26,210 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:26,210 INFO [M:0;jenkins-hbase4:38169] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38169,1691556968938; zookeeper connection closed. 2023-08-09 04:56:26,210 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): master:38169-0x101e814d6e10032, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:26,249 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-09 04:56:26,310 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:26,310 INFO [RS:0;jenkins-hbase4:39695] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39695,1691556968966; zookeeper connection closed. 2023-08-09 04:56:26,310 DEBUG [Listener at localhost/38001-EventThread] zookeeper.ZKWatcher(600): regionserver:39695-0x101e814d6e10033, quorum=127.0.0.1:63198, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-09 04:56:26,311 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@e7b1acd] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@e7b1acd 2023-08-09 04:56:26,312 INFO [Listener at localhost/38001] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-09 04:56:26,312 WARN [Listener at localhost/38001] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-09 04:56:26,316 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:26,422 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-09 04:56:26,422 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1372293509-172.31.14.131-1691556934740 (Datanode Uuid a9d3f0a9-7c3a-4851-82ac-b181ddc8a2c6) service to localhost/127.0.0.1:39381 2023-08-09 04:56:26,424 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data7/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,424 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data8/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,426 WARN [Listener at localhost/38001] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-09 04:56:26,429 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:26,536 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-09 04:56:26,536 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1372293509-172.31.14.131-1691556934740 (Datanode Uuid bc406285-cfbb-4280-a579-3ea083d46ea8) service to localhost/127.0.0.1:39381 2023-08-09 04:56:26,537 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data5/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,537 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data6/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,540 WARN [Listener at localhost/38001] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-09 04:56:26,543 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:26,648 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-09 04:56:26,648 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1372293509-172.31.14.131-1691556934740 (Datanode Uuid 35ac5fa7-7302-40f8-b02c-58995eae4684) service to localhost/127.0.0.1:39381 2023-08-09 04:56:26,648 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data3/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,649 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data4/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,650 WARN [Listener at localhost/38001] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-09 04:56:26,655 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:26,760 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-09 04:56:26,760 WARN [BP-1372293509-172.31.14.131-1691556934740 heartbeating to localhost/127.0.0.1:39381] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1372293509-172.31.14.131-1691556934740 (Datanode Uuid fe09e2dd-113c-4ad4-80f4-36ed321deac3) service to localhost/127.0.0.1:39381 2023-08-09 04:56:26,761 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data1/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,761 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/60fc02f0-1613-8a9f-6154-b32f9512523f/cluster_cf54a0cd-9565-967d-5be6-a9f89d68ef67/dfs/data/data2/current/BP-1372293509-172.31.14.131-1691556934740] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:26,794 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:26,975 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-09 04:56:26,975 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-09 04:56:26,975 WARN [Listener at localhost/38001] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-09 04:56:26,977 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:27,078 WARN [BP-140257046-172.31.14.131-1691556927574 heartbeating to localhost/127.0.0.1:45825] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-09 04:56:27,078 WARN [BP-140257046-172.31.14.131-1691556927574 heartbeating to localhost/127.0.0.1:45825] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-140257046-172.31.14.131-1691556927574 (Datanode Uuid a3b0ca79-8cf0-4760-bcdf-7f10a751d92a) service to localhost/127.0.0.1:45825 2023-08-09 04:56:27,079 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/dfs/data/data3/current/BP-140257046-172.31.14.131-1691556927574] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:27,079 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/dfs/data/data4/current/BP-140257046-172.31.14.131-1691556927574] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:27,080 WARN [Listener at localhost/38001] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-09 04:56:27,082 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:27,183 WARN [BP-140257046-172.31.14.131-1691556927574 heartbeating to localhost/127.0.0.1:45825] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-09 04:56:27,183 WARN [BP-140257046-172.31.14.131-1691556927574 heartbeating to localhost/127.0.0.1:45825] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-140257046-172.31.14.131-1691556927574 (Datanode Uuid 9dc55359-3326-4192-b6d4-feee39bb2e6c) service to localhost/127.0.0.1:45825 2023-08-09 04:56:27,184 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/dfs/data/data1/current/BP-140257046-172.31.14.131-1691556927574] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:27,184 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3ffb0b11-7101-8d17-9aaf-408cb06df4ce/cluster_6290dad5-b912-c73e-3b85-a92685690da1/dfs/data/data2/current/BP-140257046-172.31.14.131-1691556927574] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-09 04:56:27,192 INFO [Listener at localhost/38001] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-09 04:56:27,307 INFO [Listener at localhost/38001] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-09 04:56:27,331 INFO [Listener at localhost/38001] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-09 04:56:27,353 INFO [Listener at localhost/38001] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=184 (was 10) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (381036298) connection to localhost/127.0.0.1:45825 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:45825 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x41562bcd-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/38001 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x13641a94-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x4bf545ff sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/558038144.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x6774d1fc-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x4bf545ff-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:45825 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x6774d1fc sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/558038144.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41117,1691556931790-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: IPC Client (381036298) connection to localhost/127.0.0.1:39381 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x41562bcd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/558038144.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x6774d1fc-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x13641a94 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/558038144.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@674811e4 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42717,1691556961320-SendThread(127.0.0.1:63198) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:45825 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (381036298) connection to localhost/127.0.0.1:39381 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x13641a94-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x41562bcd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:45825 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (381036298) connection to localhost/127.0.0.1:39381 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (381036298) connection to localhost/127.0.0.1:45825 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:39381 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42251,1691556931838-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63198@0x4bf545ff-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) - Thread LEAK? -, OpenFileDescriptor=658 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 266), ProcessCount=171 (was 171), AvailableMemoryMB=594 (was 1824)