2023-07-19 05:55:12,044 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c 2023-07-19 05:55:12,052 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-log-dir so I do NOT create it in target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488 2023-07-19 05:55:12,052 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-log-dir Erasing configuration value by system value. 2023-07-19 05:55:12,052 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-tmp-dir so I do NOT create it in target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488 2023-07-19 05:55:12,052 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-19 05:55:12,053 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488 2023-07-19 05:55:12,066 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-19 05:55:12,067 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL timeout: 13 mins 2023-07-19 05:55:12,100 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=174, AvailableMemoryMB=7097 2023-07-19 05:55:12,111 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-log-dir so I do NOT create it in target/test-data/4d9a1560-b691-9a4a-aa2f-1bccaae10d4e 2023-07-19 05:55:12,111 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-log-dir Erasing configuration value by system value. 2023-07-19 05:55:12,112 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-tmp-dir so I do NOT create it in target/test-data/4d9a1560-b691-9a4a-aa2f-1bccaae10d4e 2023-07-19 05:55:12,112 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-19 05:55:12,112 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4d9a1560-b691-9a4a-aa2f-1bccaae10d4e 2023-07-19 05:55:12,121 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-log-dir so I do NOT create it in target/test-data/196477b1-2efc-069c-ef32-8bbfd9aced3f 2023-07-19 05:55:12,121 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-log-dir Erasing configuration value by system value. 2023-07-19 05:55:12,122 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-tmp-dir so I do NOT create it in target/test-data/196477b1-2efc-069c-ef32-8bbfd9aced3f 2023-07-19 05:55:12,122 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-19 05:55:12,122 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/196477b1-2efc-069c-ef32-8bbfd9aced3f 2023-07-19 05:55:12,126 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986, deleteOnExit=true 2023-07-19 05:55:12,244 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/zookeeper_0, clientPort=65243, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-19 05:55:12,261 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=65243 2023-07-19 05:55:12,264 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-19 05:55:12,265 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-19 05:55:12,265 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/test.cache.data in system properties and HBase conf 2023-07-19 05:55:12,265 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop.tmp.dir in system properties and HBase conf 2023-07-19 05:55:12,266 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/hadoop.log.dir in system properties and HBase conf 2023-07-19 05:55:12,266 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-19 05:55:12,266 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-19 05:55:12,267 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-19 05:55:12,368 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-19 05:55:12,726 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-19 05:55:12,730 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-19 05:55:12,730 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-19 05:55:12,731 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-19 05:55:12,731 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-19 05:55:12,732 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-19 05:55:12,732 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-19 05:55:12,733 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-19 05:55:12,733 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-19 05:55:12,734 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-19 05:55:12,734 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/nfs.dump.dir in system properties and HBase conf 2023-07-19 05:55:12,734 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/java.io.tmpdir in system properties and HBase conf 2023-07-19 05:55:12,735 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-19 05:55:12,735 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-19 05:55:12,735 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-19 05:55:13,252 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-19 05:55:13,257 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-19 05:55:13,540 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-19 05:55:13,659 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-19 05:55:13,673 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:13,711 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:13,748 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/java.io.tmpdir/Jetty_localhost_44713_hdfs____.yrmzp9/webapp 2023-07-19 05:55:13,886 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44713 2023-07-19 05:55:13,904 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-19 05:55:13,904 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-19 05:55:14,373 WARN [Listener at localhost/46083] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:14,438 WARN [Listener at localhost/46083] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-19 05:55:14,455 WARN [Listener at localhost/46083] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:14,461 INFO [Listener at localhost/46083] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:14,466 INFO [Listener at localhost/46083] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/java.io.tmpdir/Jetty_localhost_37381_datanode____5qpsvj/webapp 2023-07-19 05:55:14,563 INFO [Listener at localhost/46083] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37381 2023-07-19 05:55:14,878 WARN [Listener at localhost/37057] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:14,890 WARN [Listener at localhost/37057] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-19 05:55:14,894 WARN [Listener at localhost/37057] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:14,897 INFO [Listener at localhost/37057] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:14,904 INFO [Listener at localhost/37057] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/java.io.tmpdir/Jetty_localhost_43619_datanode____ec82g6/webapp 2023-07-19 05:55:15,006 INFO [Listener at localhost/37057] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43619 2023-07-19 05:55:15,017 WARN [Listener at localhost/33433] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:15,345 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd84b8dfbc2f550e7: Processing first storage report for DS-5a67531b-e36f-4c92-88c0-75ac8af65e90 from datanode 65e74bcf-2b26-4b98-aada-29a63cab5a77 2023-07-19 05:55:15,347 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd84b8dfbc2f550e7: from storage DS-5a67531b-e36f-4c92-88c0-75ac8af65e90 node DatanodeRegistration(127.0.0.1:41111, datanodeUuid=65e74bcf-2b26-4b98-aada-29a63cab5a77, infoPort=35889, infoSecurePort=0, ipcPort=37057, storageInfo=lv=-57;cid=testClusterID;nsid=1757840924;c=1689746113329), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-19 05:55:15,348 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd09d4c27c7796f34: Processing first storage report for DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30 from datanode 32f7061d-5342-4160-b2b9-ba7bbf92aa63 2023-07-19 05:55:15,348 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd09d4c27c7796f34: from storage DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30 node DatanodeRegistration(127.0.0.1:33877, datanodeUuid=32f7061d-5342-4160-b2b9-ba7bbf92aa63, infoPort=43807, infoSecurePort=0, ipcPort=33433, storageInfo=lv=-57;cid=testClusterID;nsid=1757840924;c=1689746113329), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:15,348 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd84b8dfbc2f550e7: Processing first storage report for DS-9b022b9a-805d-49bf-8428-62f845246d6d from datanode 65e74bcf-2b26-4b98-aada-29a63cab5a77 2023-07-19 05:55:15,348 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd84b8dfbc2f550e7: from storage DS-9b022b9a-805d-49bf-8428-62f845246d6d node DatanodeRegistration(127.0.0.1:41111, datanodeUuid=65e74bcf-2b26-4b98-aada-29a63cab5a77, infoPort=35889, infoSecurePort=0, ipcPort=37057, storageInfo=lv=-57;cid=testClusterID;nsid=1757840924;c=1689746113329), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:15,349 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd09d4c27c7796f34: Processing first storage report for DS-976a25d2-5544-4fb0-bbc6-f49795777b21 from datanode 32f7061d-5342-4160-b2b9-ba7bbf92aa63 2023-07-19 05:55:15,349 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd09d4c27c7796f34: from storage DS-976a25d2-5544-4fb0-bbc6-f49795777b21 node DatanodeRegistration(127.0.0.1:33877, datanodeUuid=32f7061d-5342-4160-b2b9-ba7bbf92aa63, infoPort=43807, infoSecurePort=0, ipcPort=33433, storageInfo=lv=-57;cid=testClusterID;nsid=1757840924;c=1689746113329), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:15,417 DEBUG [Listener at localhost/33433] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c 2023-07-19 05:55:15,428 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:15,430 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:16,103 INFO [Listener at localhost/33433] util.FSUtils(471): Created version file at hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320 with version=8 2023-07-19 05:55:16,104 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/hbase-staging 2023-07-19 05:55:16,449 INFO [Listener at localhost/33433] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-19 05:55:16,916 INFO [Listener at localhost/33433] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-19 05:55:16,947 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:16,947 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:16,948 INFO [Listener at localhost/33433] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:16,948 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:16,948 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:17,089 INFO [Listener at localhost/33433] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:17,163 DEBUG [Listener at localhost/33433] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-19 05:55:17,258 INFO [Listener at localhost/33433] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39519 2023-07-19 05:55:17,268 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,271 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,291 INFO [Listener at localhost/33433] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39519 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:17,329 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:395190x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:17,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39519-0x1017c2601c00000 connected 2023-07-19 05:55:17,354 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:55:17,355 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:17,359 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-19 05:55:17,366 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39519 2023-07-19 05:55:17,366 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39519 2023-07-19 05:55:17,367 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39519 2023-07-19 05:55:17,367 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39519 2023-07-19 05:55:17,368 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39519 2023-07-19 05:55:17,372 INFO [Listener at localhost/33433] master.HMaster(444): hbase.rootdir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320, hbase.cluster.distributed=false 2023-07-19 05:55:17,440 INFO [Listener at localhost/33433] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-19 05:55:17,440 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:17,440 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:17,440 INFO [Listener at localhost/33433] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:17,440 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:17,441 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:17,445 INFO [Listener at localhost/33433] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:17,448 INFO [Listener at localhost/33433] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43759 2023-07-19 05:55:17,451 INFO [Listener at localhost/33433] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:17,457 DEBUG [Listener at localhost/33433] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:17,458 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,460 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,461 INFO [Listener at localhost/33433] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43759 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:17,465 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:437590x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:17,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43759-0x1017c2601c00001 connected 2023-07-19 05:55:17,466 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:55:17,467 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:17,468 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-19 05:55:17,469 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43759 2023-07-19 05:55:17,469 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43759 2023-07-19 05:55:17,469 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43759 2023-07-19 05:55:17,470 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43759 2023-07-19 05:55:17,470 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43759 2023-07-19 05:55:17,483 INFO [Listener at localhost/33433] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-19 05:55:17,483 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:17,483 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:17,484 INFO [Listener at localhost/33433] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:17,484 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:17,484 INFO [Listener at localhost/33433] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:17,484 INFO [Listener at localhost/33433] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:17,486 INFO [Listener at localhost/33433] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43215 2023-07-19 05:55:17,486 INFO [Listener at localhost/33433] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:17,487 DEBUG [Listener at localhost/33433] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:17,488 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,491 INFO [Listener at localhost/33433] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,493 INFO [Listener at localhost/33433] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43215 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:17,497 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:432150x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:17,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43215-0x1017c2601c00002 connected 2023-07-19 05:55:17,498 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:55:17,499 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:17,499 DEBUG [Listener at localhost/33433] zookeeper.ZKUtil(164): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-19 05:55:17,500 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43215 2023-07-19 05:55:17,500 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43215 2023-07-19 05:55:17,500 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43215 2023-07-19 05:55:17,501 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43215 2023-07-19 05:55:17,501 DEBUG [Listener at localhost/33433] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43215 2023-07-19 05:55:17,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:17,511 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-19 05:55:17,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:17,531 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-19 05:55:17,531 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-19 05:55:17,531 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-19 05:55:17,532 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:17,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-19 05:55:17,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,39519,1689746116265 from backup master directory 2023-07-19 05:55:17,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-19 05:55:17,537 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:17,537 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-19 05:55:17,538 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:17,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:17,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-19 05:55:17,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-19 05:55:17,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/hbase.id with ID: fa474148-f7f6-4a32-a2c9-9f3f924e734e 2023-07-19 05:55:17,680 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:17,697 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:17,741 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x08e0a71d to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:17,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6794ff67, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:17,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:17,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-19 05:55:17,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:17,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:17,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store-tmp 2023-07-19 05:55:17,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:17,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:55:17,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:17,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:17,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:55:17,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:17,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:17,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:17,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:17,883 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:17,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/oldWALs, maxLogs=10 2023-07-19 05:55:17,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-19 05:55:17,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-19 05:55:17,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-19 05:55:17,987 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-19 05:55:17,990 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:17,998 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:18,001 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-19 05:55:18,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 2023-07-19 05:55:18,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK], DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK]] 2023-07-19 05:55:18,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:18,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:18,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:18,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:18,146 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:18,154 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-19 05:55:18,178 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-19 05:55:18,192 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:18,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:18,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:18,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:18,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:18,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68510160, jitterRate=0.020880937576293945}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:18,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:18,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-19 05:55:18,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-19 05:55:18,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-19 05:55:18,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-19 05:55:18,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-19 05:55:18,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 35 msec 2023-07-19 05:55:18,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-19 05:55:18,314 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-19 05:55:18,320 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-19 05:55:18,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-19 05:55:18,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-19 05:55:18,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-19 05:55:18,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-19 05:55:18,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-19 05:55:18,369 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:18,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-19 05:55:18,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-19 05:55:18,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-19 05:55:18,394 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-19 05:55:18,394 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-19 05:55:18,394 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-19 05:55:18,394 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:18,395 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39519,1689746116265, sessionid=0x1017c2601c00000, setting cluster-up flag (Was=false) 2023-07-19 05:55:18,414 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:18,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-19 05:55:18,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:18,429 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:18,435 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-19 05:55:18,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:18,442 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.hbase-snapshot/.tmp 2023-07-19 05:55:18,507 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(951): ClusterId : fa474148-f7f6-4a32-a2c9-9f3f924e734e 2023-07-19 05:55:18,507 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(951): ClusterId : fa474148-f7f6-4a32-a2c9-9f3f924e734e 2023-07-19 05:55:18,513 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:18,513 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:18,521 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:18,521 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:18,521 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:18,521 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:18,525 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:18,526 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:18,526 DEBUG [RS:1;jenkins-hbase4:43215] zookeeper.ReadOnlyZKClient(139): Connect 0x29f89bf0 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:18,527 DEBUG [RS:0;jenkins-hbase4:43759] zookeeper.ReadOnlyZKClient(139): Connect 0x4458162a to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:18,538 DEBUG [RS:1;jenkins-hbase4:43215] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@273f70bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:18,546 DEBUG [RS:1;jenkins-hbase4:43215] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2300f080, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:18,549 DEBUG [RS:0;jenkins-hbase4:43759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a9ff0d1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:18,549 DEBUG [RS:0;jenkins-hbase4:43759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ea07d5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:18,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:18,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:18,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,575 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:43215 2023-07-19 05:55:18,578 INFO [RS:1;jenkins-hbase4:43215] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:18,578 INFO [RS:1;jenkins-hbase4:43215] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:18,578 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:18,575 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43759 2023-07-19 05:55:18,579 INFO [RS:0;jenkins-hbase4:43759] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:18,584 INFO [RS:0;jenkins-hbase4:43759] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:18,584 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:18,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689746148586 2023-07-19 05:55:18,588 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39519,1689746116265 with isa=jenkins-hbase4.apache.org/172.31.14.131:43215, startcode=1689746117482 2023-07-19 05:55:18,588 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39519,1689746116265 with isa=jenkins-hbase4.apache.org/172.31.14.131:43759, startcode=1689746117439 2023-07-19 05:55:18,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-19 05:55:18,594 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:18,594 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-19 05:55:18,599 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:18,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-19 05:55:18,610 DEBUG [RS:0;jenkins-hbase4:43759] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:18,610 DEBUG [RS:1;jenkins-hbase4:43215] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:18,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-19 05:55:18,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-19 05:55:18,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-19 05:55:18,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-19 05:55:18,616 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,618 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-19 05:55:18,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-19 05:55:18,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-19 05:55:18,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-19 05:55:18,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-19 05:55:18,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746118636,5,FailOnTimeoutGroup] 2023-07-19 05:55:18,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746118639,5,FailOnTimeoutGroup] 2023-07-19 05:55:18,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-19 05:55:18,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,649 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:18,650 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:18,651 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320 2023-07-19 05:55:18,681 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:18,684 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:55:18,691 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51599, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:18,691 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51105, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:18,693 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info 2023-07-19 05:55:18,694 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:55:18,695 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:18,695 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:55:18,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:18,700 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:55:18,701 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:18,701 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:55:18,703 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39519] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,704 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table 2023-07-19 05:55:18,705 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:55:18,706 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:18,708 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740 2023-07-19 05:55:18,709 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740 2023-07-19 05:55:18,713 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:55:18,715 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:55:18,719 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39519] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,721 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:18,722 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62346283, jitterRate=-0.07096798717975616}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:55:18,722 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:55:18,722 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:55:18,722 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:55:18,722 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:55:18,722 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:55:18,723 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:55:18,723 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:55:18,724 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:55:18,725 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320 2023-07-19 05:55:18,725 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320 2023-07-19 05:55:18,725 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46083 2023-07-19 05:55:18,725 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46083 2023-07-19 05:55:18,725 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:18,725 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:18,730 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:18,730 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-19 05:55:18,731 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:55:18,732 DEBUG [RS:1;jenkins-hbase4:43215] zookeeper.ZKUtil(162): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,732 WARN [RS:1;jenkins-hbase4:43215] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:18,733 DEBUG [RS:0;jenkins-hbase4:43759] zookeeper.ZKUtil(162): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,733 INFO [RS:1;jenkins-hbase4:43215] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:18,734 WARN [RS:0;jenkins-hbase4:43759] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:18,734 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:18,734 INFO [RS:1;jenkins-hbase4:43215] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:18,734 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43759,1689746117439] 2023-07-19 05:55:18,735 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,734 INFO [RS:0;jenkins-hbase4:43759] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:18,735 INFO [RS:0;jenkins-hbase4:43759] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:18,735 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-19 05:55:18,747 DEBUG [RS:1;jenkins-hbase4:43215] zookeeper.ZKUtil(162): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,747 DEBUG [RS:0;jenkins-hbase4:43759] zookeeper.ZKUtil(162): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,747 DEBUG [RS:1;jenkins-hbase4:43215] zookeeper.ZKUtil(162): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,748 DEBUG [RS:0;jenkins-hbase4:43759] zookeeper.ZKUtil(162): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,758 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-19 05:55:18,759 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:18,759 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:18,761 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-19 05:55:18,769 INFO [RS:0;jenkins-hbase4:43759] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:18,779 INFO [RS:1;jenkins-hbase4:43215] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:18,799 INFO [RS:0;jenkins-hbase4:43759] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:18,799 INFO [RS:1;jenkins-hbase4:43215] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:18,803 INFO [RS:0;jenkins-hbase4:43759] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:18,803 INFO [RS:1;jenkins-hbase4:43215] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:18,803 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,803 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,804 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:18,804 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:18,814 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,814 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,814 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,814 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,814 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,814 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,814 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:18,815 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,815 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,816 DEBUG [RS:0;jenkins-hbase4:43759] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,816 DEBUG [RS:1;jenkins-hbase4:43215] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:18,821 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,821 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,821 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,826 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,826 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,826 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,842 INFO [RS:0;jenkins-hbase4:43759] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:18,842 INFO [RS:1;jenkins-hbase4:43215] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:18,845 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43759,1689746117439-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,845 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43215,1689746117482-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:18,876 INFO [RS:1;jenkins-hbase4:43215] regionserver.Replication(203): jenkins-hbase4.apache.org,43215,1689746117482 started 2023-07-19 05:55:18,876 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43215,1689746117482, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43215, sessionid=0x1017c2601c00002 2023-07-19 05:55:18,876 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:18,876 DEBUG [RS:1;jenkins-hbase4:43215] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,876 DEBUG [RS:1;jenkins-hbase4:43215] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43215,1689746117482' 2023-07-19 05:55:18,876 DEBUG [RS:1;jenkins-hbase4:43215] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-19 05:55:18,877 INFO [RS:0;jenkins-hbase4:43759] regionserver.Replication(203): jenkins-hbase4.apache.org,43759,1689746117439 started 2023-07-19 05:55:18,877 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43759,1689746117439, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43759, sessionid=0x1017c2601c00001 2023-07-19 05:55:18,877 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:18,877 DEBUG [RS:0;jenkins-hbase4:43759] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,878 DEBUG [RS:0;jenkins-hbase4:43759] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43759,1689746117439' 2023-07-19 05:55:18,878 DEBUG [RS:0;jenkins-hbase4:43759] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-19 05:55:18,879 DEBUG [RS:1;jenkins-hbase4:43215] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-19 05:55:18,879 DEBUG [RS:0;jenkins-hbase4:43759] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-19 05:55:18,880 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:18,880 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:18,880 DEBUG [RS:1;jenkins-hbase4:43215] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:18,880 DEBUG [RS:1;jenkins-hbase4:43215] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43215,1689746117482' 2023-07-19 05:55:18,880 DEBUG [RS:1;jenkins-hbase4:43215] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-19 05:55:18,880 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:18,880 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:18,880 DEBUG [RS:0;jenkins-hbase4:43759] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:18,881 DEBUG [RS:0;jenkins-hbase4:43759] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43759,1689746117439' 2023-07-19 05:55:18,881 DEBUG [RS:0;jenkins-hbase4:43759] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-19 05:55:18,881 DEBUG [RS:1;jenkins-hbase4:43215] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-19 05:55:18,881 DEBUG [RS:0;jenkins-hbase4:43759] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-19 05:55:18,882 DEBUG [RS:1;jenkins-hbase4:43215] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:18,882 INFO [RS:1;jenkins-hbase4:43215] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:18,882 INFO [RS:1;jenkins-hbase4:43215] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:18,882 DEBUG [RS:0;jenkins-hbase4:43759] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:18,882 INFO [RS:0;jenkins-hbase4:43759] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:18,882 INFO [RS:0;jenkins-hbase4:43759] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:18,913 DEBUG [jenkins-hbase4:39519] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-19 05:55:18,917 DEBUG [jenkins-hbase4:39519] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:18,923 DEBUG [jenkins-hbase4:39519] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:18,923 DEBUG [jenkins-hbase4:39519] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:18,923 DEBUG [jenkins-hbase4:39519] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:18,926 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43759,1689746117439, state=OPENING 2023-07-19 05:55:18,938 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-19 05:55:18,939 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:18,940 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-19 05:55:18,945 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43759,1689746117439}] 2023-07-19 05:55:18,989 INFO [RS:1;jenkins-hbase4:43215] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:18,989 INFO [RS:0;jenkins-hbase4:43759] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:18,993 INFO [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:18,993 INFO [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:19,022 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:19,023 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:19,025 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:19,035 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:19,037 INFO [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 2023-07-19 05:55:19,043 DEBUG [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:55:19,045 INFO [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 2023-07-19 05:55:19,045 DEBUG [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK], DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK]] 2023-07-19 05:55:19,129 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:19,131 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:19,136 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34318, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:19,150 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-19 05:55:19,151 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:19,151 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:19,151 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:19,157 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43759%2C1689746117439.meta, suffix=.meta, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:19,178 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:19,178 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:19,185 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.meta.1689746119158.meta 2023-07-19 05:55:19,186 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK], DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK]] 2023-07-19 05:55:19,186 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:19,189 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-19 05:55:19,207 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-19 05:55:19,242 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-19 05:55:19,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-19 05:55:19,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:19,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-19 05:55:19,248 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-19 05:55:19,251 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:55:19,253 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info 2023-07-19 05:55:19,253 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info 2023-07-19 05:55:19,254 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:55:19,255 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:19,255 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:55:19,256 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:19,256 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:19,257 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:55:19,258 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:19,258 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:55:19,260 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table 2023-07-19 05:55:19,260 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table 2023-07-19 05:55:19,260 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:55:19,261 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:19,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740 2023-07-19 05:55:19,266 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740 2023-07-19 05:55:19,270 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:55:19,275 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:55:19,277 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62312862, jitterRate=-0.07146599888801575}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:55:19,277 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:55:19,287 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689746119121 2023-07-19 05:55:19,310 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-19 05:55:19,310 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-19 05:55:19,311 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43759,1689746117439, state=OPEN 2023-07-19 05:55:19,314 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-19 05:55:19,314 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-19 05:55:19,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-19 05:55:19,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43759,1689746117439 in 369 msec 2023-07-19 05:55:19,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-19 05:55:19,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 576 msec 2023-07-19 05:55:19,330 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 843 msec 2023-07-19 05:55:19,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689746119330, completionTime=-1 2023-07-19 05:55:19,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-19 05:55:19,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-19 05:55:19,419 DEBUG [hconnection-0x5e98d029-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:19,423 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34324, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:19,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-19 05:55:19,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689746179457 2023-07-19 05:55:19,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689746239457 2023-07-19 05:55:19,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 125 msec 2023-07-19 05:55:19,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39519,1689746116265-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:19,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39519,1689746116265-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:19,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39519,1689746116265-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:19,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39519, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:19,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:19,493 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-19 05:55:19,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-19 05:55:19,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:19,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-19 05:55:19,521 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-19 05:55:19,524 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-19 05:55:19,547 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:19,549 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 empty. 2023-07-19 05:55:19,550 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:19,550 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-19 05:55:19,580 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:19,582 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 13c92efbf45fed23cebac57cf655e861, NAME => 'hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp 2023-07-19 05:55:19,599 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:19,599 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 13c92efbf45fed23cebac57cf655e861, disabling compactions & flushes 2023-07-19 05:55:19,599 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:19,599 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:19,599 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. after waiting 0 ms 2023-07-19 05:55:19,599 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:19,599 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:19,599 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 13c92efbf45fed23cebac57cf655e861: 2023-07-19 05:55:19,606 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-19 05:55:19,626 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746119609"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689746119609"}]},"ts":"1689746119609"} 2023-07-19 05:55:19,659 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-19 05:55:19,661 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-19 05:55:19,665 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746119661"}]},"ts":"1689746119661"} 2023-07-19 05:55:19,668 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-19 05:55:19,672 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:19,674 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:19,674 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:19,675 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:19,677 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN}] 2023-07-19 05:55:19,679 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN 2023-07-19 05:55:19,681 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43215,1689746117482; forceNewPlan=false, retain=false 2023-07-19 05:55:19,833 INFO [jenkins-hbase4:39519] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-19 05:55:19,834 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=13c92efbf45fed23cebac57cf655e861, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:19,834 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746119834"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746119834"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746119834"}]},"ts":"1689746119834"} 2023-07-19 05:55:19,839 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 13c92efbf45fed23cebac57cf655e861, server=jenkins-hbase4.apache.org,43215,1689746117482}] 2023-07-19 05:55:19,993 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:19,994 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:19,997 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:20,003 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:20,003 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:20,007 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:20,026 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:20,026 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:20,031 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008 2023-07-19 05:55:20,031 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:55:20,032 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 13c92efbf45fed23cebac57cf655e861, NAME => 'hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:20,032 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:20,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,033 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,037 INFO [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,040 DEBUG [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info 2023-07-19 05:55:20,040 DEBUG [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info 2023-07-19 05:55:20,040 INFO [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 13c92efbf45fed23cebac57cf655e861 columnFamilyName info 2023-07-19 05:55:20,041 INFO [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] regionserver.HStore(310): Store=13c92efbf45fed23cebac57cf655e861/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:20,043 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,044 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,049 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:20,055 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:20,056 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 13c92efbf45fed23cebac57cf655e861; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70589307, jitterRate=0.05186264216899872}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:20,056 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 13c92efbf45fed23cebac57cf655e861: 2023-07-19 05:55:20,058 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861., pid=6, masterSystemTime=1689746119993 2023-07-19 05:55:20,063 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:20,064 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:20,065 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=13c92efbf45fed23cebac57cf655e861, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:20,065 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746120064"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746120064"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746120064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746120064"}]},"ts":"1689746120064"} 2023-07-19 05:55:20,073 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-19 05:55:20,073 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 13c92efbf45fed23cebac57cf655e861, server=jenkins-hbase4.apache.org,43215,1689746117482 in 230 msec 2023-07-19 05:55:20,076 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-19 05:55:20,077 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN in 396 msec 2023-07-19 05:55:20,078 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-19 05:55:20,078 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746120078"}]},"ts":"1689746120078"} 2023-07-19 05:55:20,081 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-19 05:55:20,085 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-19 05:55:20,087 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 577 msec 2023-07-19 05:55:20,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-19 05:55:20,124 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-19 05:55:20,124 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:20,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:20,154 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49712, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:20,170 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-19 05:55:20,199 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-19 05:55:20,205 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 44 msec 2023-07-19 05:55:20,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-19 05:55:20,225 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-19 05:55:20,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-07-19 05:55:20,241 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-19 05:55:20,244 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-19 05:55:20,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.706sec 2023-07-19 05:55:20,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-19 05:55:20,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-19 05:55:20,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-19 05:55:20,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39519,1689746116265-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-19 05:55:20,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39519,1689746116265-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-19 05:55:20,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-19 05:55:20,312 DEBUG [Listener at localhost/33433] zookeeper.ReadOnlyZKClient(139): Connect 0x75fbe421 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:20,319 DEBUG [Listener at localhost/33433] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@522f9b44, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:20,333 DEBUG [hconnection-0x70c6f597-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:20,345 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40886, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:20,357 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:20,357 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-19 05:55:20,357 INFO [Listener at localhost/33433] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e, deleteOnExit=true 2023-07-19 05:55:20,357 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-19 05:55:20,368 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/test.cache.data in system properties and HBase conf 2023-07-19 05:55:20,369 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/hadoop.tmp.dir in system properties and HBase conf 2023-07-19 05:55:20,369 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/hadoop.log.dir in system properties and HBase conf 2023-07-19 05:55:20,369 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-19 05:55:20,369 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-19 05:55:20,369 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-19 05:55:20,369 DEBUG [Listener at localhost/33433] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-19 05:55:20,370 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-19 05:55:20,370 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-19 05:55:20,371 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-19 05:55:20,371 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-19 05:55:20,371 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-19 05:55:20,371 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-19 05:55:20,371 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-19 05:55:20,372 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-19 05:55:20,372 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-19 05:55:20,372 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/nfs.dump.dir in system properties and HBase conf 2023-07-19 05:55:20,372 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/java.io.tmpdir in system properties and HBase conf 2023-07-19 05:55:20,372 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-19 05:55:20,373 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-19 05:55:20,373 INFO [Listener at localhost/33433] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-19 05:55:20,381 WARN [Listener at localhost/33433] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-19 05:55:20,382 WARN [Listener at localhost/33433] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-19 05:55:20,437 WARN [Listener at localhost/33433] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:20,439 INFO [Listener at localhost/33433] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:20,444 INFO [Listener at localhost/33433] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/java.io.tmpdir/Jetty_localhost_35561_hdfs____45qb3s/webapp 2023-07-19 05:55:20,542 INFO [Listener at localhost/33433] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35561 2023-07-19 05:55:20,554 WARN [Listener at localhost/33433] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-19 05:55:20,554 WARN [Listener at localhost/33433] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-19 05:55:20,615 WARN [Listener at localhost/33839] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:20,645 WARN [Listener at localhost/33839] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-19 05:55:20,648 WARN [Listener at localhost/33839] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:20,650 INFO [Listener at localhost/33839] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:20,658 INFO [Listener at localhost/33839] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/java.io.tmpdir/Jetty_localhost_42887_datanode____ntovog/webapp 2023-07-19 05:55:20,779 INFO [Listener at localhost/33839] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42887 2023-07-19 05:55:20,789 WARN [Listener at localhost/33597] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:20,831 WARN [Listener at localhost/33597] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-19 05:55:20,834 WARN [Listener at localhost/33597] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:20,836 INFO [Listener at localhost/33597] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:20,844 INFO [Listener at localhost/33597] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/java.io.tmpdir/Jetty_localhost_40585_datanode____.odozs3/webapp 2023-07-19 05:55:21,031 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xed1c4adbb9a350ca: Processing first storage report for DS-1adb813e-2575-47a2-84ac-2cef5ee08feb from datanode 31c61bf0-0d47-4d7e-8864-33d9f690413c 2023-07-19 05:55:21,033 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xed1c4adbb9a350ca: from storage DS-1adb813e-2575-47a2-84ac-2cef5ee08feb node DatanodeRegistration(127.0.0.1:33225, datanodeUuid=31c61bf0-0d47-4d7e-8864-33d9f690413c, infoPort=39119, infoSecurePort=0, ipcPort=33597, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,033 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xed1c4adbb9a350ca: Processing first storage report for DS-72ac0f7c-07ed-4811-8ed4-538c10c9217e from datanode 31c61bf0-0d47-4d7e-8864-33d9f690413c 2023-07-19 05:55:21,033 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xed1c4adbb9a350ca: from storage DS-72ac0f7c-07ed-4811-8ed4-538c10c9217e node DatanodeRegistration(127.0.0.1:33225, datanodeUuid=31c61bf0-0d47-4d7e-8864-33d9f690413c, infoPort=39119, infoSecurePort=0, ipcPort=33597, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,086 INFO [Listener at localhost/33597] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40585 2023-07-19 05:55:21,101 WARN [Listener at localhost/38307] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:21,130 WARN [Listener at localhost/38307] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-19 05:55:21,133 WARN [Listener at localhost/38307] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:21,134 INFO [Listener at localhost/38307] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:21,143 INFO [Listener at localhost/38307] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/java.io.tmpdir/Jetty_localhost_40869_datanode____.navo9m/webapp 2023-07-19 05:55:21,242 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x438769902e8eaa54: Processing first storage report for DS-35686ffa-ada0-46f2-b5b2-f8da3b859896 from datanode 5e7adc69-4203-4bea-b585-ecb199f232f2 2023-07-19 05:55:21,242 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x438769902e8eaa54: from storage DS-35686ffa-ada0-46f2-b5b2-f8da3b859896 node DatanodeRegistration(127.0.0.1:35049, datanodeUuid=5e7adc69-4203-4bea-b585-ecb199f232f2, infoPort=39027, infoSecurePort=0, ipcPort=38307, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,242 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x438769902e8eaa54: Processing first storage report for DS-cb4f6f7b-2e26-495f-91d9-614978838679 from datanode 5e7adc69-4203-4bea-b585-ecb199f232f2 2023-07-19 05:55:21,242 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x438769902e8eaa54: from storage DS-cb4f6f7b-2e26-495f-91d9-614978838679 node DatanodeRegistration(127.0.0.1:35049, datanodeUuid=5e7adc69-4203-4bea-b585-ecb199f232f2, infoPort=39027, infoSecurePort=0, ipcPort=38307, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,273 INFO [Listener at localhost/38307] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40869 2023-07-19 05:55:21,282 WARN [Listener at localhost/36575] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:21,332 WARN [Listener at localhost/36575] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-19 05:55:21,342 WARN [Listener at localhost/36575] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-19 05:55:21,344 INFO [Listener at localhost/36575] log.Slf4jLog(67): jetty-6.1.26 2023-07-19 05:55:21,357 INFO [Listener at localhost/36575] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/java.io.tmpdir/Jetty_localhost_43259_datanode____n88ela/webapp 2023-07-19 05:55:21,445 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x33a181ca3cdac026: Processing first storage report for DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e from datanode a5183c54-915c-49cb-a0fd-a74b2feb6fb4 2023-07-19 05:55:21,445 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x33a181ca3cdac026: from storage DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e node DatanodeRegistration(127.0.0.1:37059, datanodeUuid=a5183c54-915c-49cb-a0fd-a74b2feb6fb4, infoPort=35867, infoSecurePort=0, ipcPort=36575, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,445 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x33a181ca3cdac026: Processing first storage report for DS-98bbac7a-ca62-498a-9b20-af6c50abb8a3 from datanode a5183c54-915c-49cb-a0fd-a74b2feb6fb4 2023-07-19 05:55:21,445 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x33a181ca3cdac026: from storage DS-98bbac7a-ca62-498a-9b20-af6c50abb8a3 node DatanodeRegistration(127.0.0.1:37059, datanodeUuid=a5183c54-915c-49cb-a0fd-a74b2feb6fb4, infoPort=35867, infoSecurePort=0, ipcPort=36575, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,490 INFO [Listener at localhost/36575] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43259 2023-07-19 05:55:21,499 WARN [Listener at localhost/45003] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-19 05:55:21,613 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x40185c4c8c3b7387: Processing first storage report for DS-9a06766e-3818-433b-ad90-54a618b3c648 from datanode 7da89a07-25bf-4d11-b4fa-0f0716110233 2023-07-19 05:55:21,613 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x40185c4c8c3b7387: from storage DS-9a06766e-3818-433b-ad90-54a618b3c648 node DatanodeRegistration(127.0.0.1:43005, datanodeUuid=7da89a07-25bf-4d11-b4fa-0f0716110233, infoPort=40355, infoSecurePort=0, ipcPort=45003, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,614 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x40185c4c8c3b7387: Processing first storage report for DS-c1642424-a558-441a-bdb3-06d3f56fc77d from datanode 7da89a07-25bf-4d11-b4fa-0f0716110233 2023-07-19 05:55:21,614 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x40185c4c8c3b7387: from storage DS-c1642424-a558-441a-bdb3-06d3f56fc77d node DatanodeRegistration(127.0.0.1:43005, datanodeUuid=7da89a07-25bf-4d11-b4fa-0f0716110233, infoPort=40355, infoSecurePort=0, ipcPort=45003, storageInfo=lv=-57;cid=testClusterID;nsid=544000682;c=1689746120389), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-19 05:55:21,717 DEBUG [Listener at localhost/45003] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488 2023-07-19 05:55:21,717 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,718 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,765 INFO [Listener at localhost/45003] util.FSUtils(471): Created version file at hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 with version=8 2023-07-19 05:55:21,766 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/hbase-staging 2023-07-19 05:55:21,768 INFO [Listener at localhost/45003] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:21,768 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,768 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,768 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:21,768 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,769 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:21,769 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:21,772 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36183 2023-07-19 05:55:21,773 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,775 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,776 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36183 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:21,781 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:361830x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:21,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36183-0x1017c2601c00007 connected 2023-07-19 05:55:21,802 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:21,803 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:21,804 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:21,804 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36183 2023-07-19 05:55:21,804 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36183 2023-07-19 05:55:21,804 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36183 2023-07-19 05:55:21,805 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36183 2023-07-19 05:55:21,805 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36183 2023-07-19 05:55:21,805 INFO [Listener at localhost/45003] master.HMaster(444): hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881, hbase.cluster.distributed=false 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:21,820 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:21,822 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45061 2023-07-19 05:55:21,823 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:21,824 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:21,825 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,826 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,828 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45061 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:21,832 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:450610x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:21,833 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:450610x0, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:21,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45061-0x1017c2601c00008 connected 2023-07-19 05:55:21,834 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:21,835 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:21,839 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45061 2023-07-19 05:55:21,839 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45061 2023-07-19 05:55:21,839 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45061 2023-07-19 05:55:21,842 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45061 2023-07-19 05:55:21,842 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45061 2023-07-19 05:55:21,857 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:21,857 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,857 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,857 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:21,857 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,858 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:21,858 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:21,860 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34613 2023-07-19 05:55:21,860 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:21,861 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:21,862 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,863 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,864 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34613 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:21,870 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:346130x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:21,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34613-0x1017c2601c00009 connected 2023-07-19 05:55:21,872 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:21,872 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:21,873 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:21,873 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34613 2023-07-19 05:55:21,874 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34613 2023-07-19 05:55:21,874 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34613 2023-07-19 05:55:21,874 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34613 2023-07-19 05:55:21,875 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34613 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:21,887 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:21,889 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43495 2023-07-19 05:55:21,890 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:21,891 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:21,891 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,893 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,894 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43495 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:21,898 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:434950x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:21,899 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:434950x0, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:21,900 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:434950x0, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:21,901 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:434950x0, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:21,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43495-0x1017c2601c0000a connected 2023-07-19 05:55:21,922 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43495 2023-07-19 05:55:21,922 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43495 2023-07-19 05:55:21,923 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43495 2023-07-19 05:55:21,924 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43495 2023-07-19 05:55:21,924 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43495 2023-07-19 05:55:21,941 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:21,941 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,942 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,942 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:21,942 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:21,942 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:21,942 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:21,948 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42043 2023-07-19 05:55:21,948 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:21,949 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:21,950 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,952 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:21,953 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42043 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:21,957 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:420430x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:21,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42043-0x1017c2601c0000b connected 2023-07-19 05:55:21,959 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:21,960 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:21,963 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:21,963 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42043 2023-07-19 05:55:21,964 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42043 2023-07-19 05:55:21,964 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42043 2023-07-19 05:55:21,967 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42043 2023-07-19 05:55:21,968 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42043 2023-07-19 05:55:21,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:21,972 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-19 05:55:21,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:21,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:21,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:21,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:21,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:21,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:21,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:21,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-19 05:55:21,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-19 05:55:21,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,36183,1689746121767 from backup master directory 2023-07-19 05:55:21,986 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:21,986 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-19 05:55:21,986 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:21,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:22,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/hbase.id with ID: 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:22,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:22,046 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:22,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3d93b016 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:22,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a4d7720, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:22,087 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:22,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-19 05:55:22,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:22,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:22,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store-tmp 2023-07-19 05:55:22,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:22,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:55:22,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:22,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:22,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:55:22,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:22,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:22,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:22,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:22,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:22,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/oldWALs, maxLogs=10 2023-07-19 05:55:22,152 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:22,155 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:22,156 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:22,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 2023-07-19 05:55:22,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK]] 2023-07-19 05:55:22,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:22,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:22,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:22,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:22,172 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:22,175 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-19 05:55:22,175 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-19 05:55:22,176 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,178 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:22,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:22,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:22,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:22,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62912659, jitterRate=-0.0625283271074295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:22,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:22,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-19 05:55:22,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-19 05:55:22,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-19 05:55:22,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-19 05:55:22,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-19 05:55:22,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-19 05:55:22,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-19 05:55:22,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-19 05:55:22,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-19 05:55:22,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-19 05:55:22,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-19 05:55:22,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-19 05:55:22,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-19 05:55:22,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-19 05:55:22,207 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:22,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-19 05:55:22,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-19 05:55:22,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-19 05:55:22,212 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:22,212 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:22,212 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:22,212 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:22,212 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:22,212 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:22,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,36183,1689746121767, sessionid=0x1017c2601c00007, setting cluster-up flag (Was=false) 2023-07-19 05:55:22,218 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:22,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-19 05:55:22,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:22,232 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:22,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-19 05:55:22,240 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:22,241 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.hbase-snapshot/.tmp 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:22,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689746152246 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-19 05:55:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-19 05:55:22,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-19 05:55:22,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-19 05:55:22,249 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:22,249 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-19 05:55:22,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-19 05:55:22,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-19 05:55:22,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746122249,5,FailOnTimeoutGroup] 2023-07-19 05:55:22,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746122249,5,FailOnTimeoutGroup] 2023-07-19 05:55:22,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-19 05:55:22,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,251 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:22,271 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(951): ClusterId : 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:22,271 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(951): ClusterId : 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:22,277 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:22,271 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(951): ClusterId : 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:22,279 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:22,280 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:22,283 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(951): ClusterId : 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:22,284 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:22,285 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:22,286 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:22,286 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:22,286 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:22,286 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:22,287 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:22,287 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:55:22,289 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:22,289 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:22,294 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:22,294 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:22,294 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:22,298 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:22,299 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:22,302 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ReadOnlyZKClient(139): Connect 0x29d5555b to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:22,302 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ReadOnlyZKClient(139): Connect 0x28660803 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:22,304 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ReadOnlyZKClient(139): Connect 0x1f58b9d2 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:22,314 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:22,321 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ReadOnlyZKClient(139): Connect 0x00e2710f to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:22,333 DEBUG [RS:0;jenkins-hbase4:45061] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@26a1f37f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:22,335 DEBUG [RS:1;jenkins-hbase4:34613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@232d3e2b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:22,335 DEBUG [RS:0;jenkins-hbase4:45061] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@679cf944, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:22,336 DEBUG [RS:1;jenkins-hbase4:34613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@662fa42a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:22,342 DEBUG [RS:2;jenkins-hbase4:43495] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b672ee1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:22,343 DEBUG [RS:2;jenkins-hbase4:43495] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e65b3e9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:22,343 DEBUG [RS:3;jenkins-hbase4:42043] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e258f63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:22,343 DEBUG [RS:3;jenkins-hbase4:42043] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a1a1a7f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:22,353 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34613 2023-07-19 05:55:22,353 INFO [RS:1;jenkins-hbase4:34613] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:22,353 INFO [RS:1;jenkins-hbase4:34613] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:22,353 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:22,354 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36183,1689746121767 with isa=jenkins-hbase4.apache.org/172.31.14.131:34613, startcode=1689746121857 2023-07-19 05:55:22,354 DEBUG [RS:1;jenkins-hbase4:34613] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:22,355 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:42043 2023-07-19 05:55:22,355 INFO [RS:3;jenkins-hbase4:42043] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:22,355 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:43495 2023-07-19 05:55:22,355 INFO [RS:3;jenkins-hbase4:42043] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:22,356 INFO [RS:2;jenkins-hbase4:43495] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:22,356 INFO [RS:2;jenkins-hbase4:43495] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:22,356 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:22,356 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:22,357 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36183,1689746121767 with isa=jenkins-hbase4.apache.org/172.31.14.131:42043, startcode=1689746121940 2023-07-19 05:55:22,357 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36183,1689746121767 with isa=jenkins-hbase4.apache.org/172.31.14.131:43495, startcode=1689746121886 2023-07-19 05:55:22,357 DEBUG [RS:3;jenkins-hbase4:42043] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:22,359 DEBUG [RS:2;jenkins-hbase4:43495] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:22,357 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45061 2023-07-19 05:55:22,360 INFO [RS:0;jenkins-hbase4:45061] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:22,360 INFO [RS:0;jenkins-hbase4:45061] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:22,360 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:22,362 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47927, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:22,363 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:22,363 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36183,1689746121767 with isa=jenkins-hbase4.apache.org/172.31.14.131:45061, startcode=1689746121819 2023-07-19 05:55:22,364 DEBUG [RS:0;jenkins-hbase4:45061] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:22,367 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47745, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:22,371 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:55:22,374 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36183] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,375 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50943, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:22,375 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38495, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:22,375 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36183] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,375 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:55:22,375 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33839 2023-07-19 05:55:22,375 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36183] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,375 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:55:22,375 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:22,376 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33839 2023-07-19 05:55:22,376 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:22,376 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36183] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,376 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:55:22,377 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:55:22,377 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33839 2023-07-19 05:55:22,377 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:22,377 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:55:22,377 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:55:22,377 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33839 2023-07-19 05:55:22,377 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:22,378 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,379 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:55:22,379 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:22,381 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ZKUtil(162): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,381 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ZKUtil(162): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,381 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42043,1689746121940] 2023-07-19 05:55:22,381 WARN [RS:1;jenkins-hbase4:34613] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:22,382 INFO [RS:1;jenkins-hbase4:34613] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:22,382 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ZKUtil(162): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,382 WARN [RS:3;jenkins-hbase4:42043] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:22,382 INFO [RS:3;jenkins-hbase4:42043] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:22,382 INFO [RS:3;jenkins-hbase4:42043] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:22,382 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,382 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34613,1689746121857] 2023-07-19 05:55:22,381 WARN [RS:0;jenkins-hbase4:45061] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:22,382 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45061,1689746121819] 2023-07-19 05:55:22,382 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43495,1689746121886] 2023-07-19 05:55:22,382 INFO [RS:1;jenkins-hbase4:34613] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:22,382 INFO [RS:0;jenkins-hbase4:45061] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:22,382 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:22,382 INFO [RS:0;jenkins-hbase4:45061] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:22,382 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,383 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,383 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:55:22,384 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ZKUtil(162): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,384 WARN [RS:2;jenkins-hbase4:43495] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:22,384 INFO [RS:2;jenkins-hbase4:43495] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:22,384 INFO [RS:2;jenkins-hbase4:43495] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:22,384 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,385 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,388 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:55:22,396 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:55:22,397 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:55:22,397 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ZKUtil(162): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,398 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,400 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ZKUtil(162): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,400 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ZKUtil(162): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,400 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ZKUtil(162): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,400 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ZKUtil(162): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,400 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ZKUtil(162): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,400 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:55:22,400 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ZKUtil(162): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,401 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ZKUtil(162): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,401 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ZKUtil(162): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,401 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ZKUtil(162): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,401 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:55:22,402 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ZKUtil(162): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,402 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ZKUtil(162): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,402 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ZKUtil(162): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,402 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ZKUtil(162): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,403 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:22,403 INFO [RS:0;jenkins-hbase4:45061] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:22,403 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ZKUtil(162): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,404 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ZKUtil(162): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,406 DEBUG [RS:1;jenkins-hbase4:34613] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:22,406 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:22,406 INFO [RS:1;jenkins-hbase4:34613] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:22,406 INFO [RS:0;jenkins-hbase4:45061] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:22,406 INFO [RS:2;jenkins-hbase4:43495] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:22,407 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:22,407 INFO [RS:0;jenkins-hbase4:45061] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:22,408 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,408 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:55:22,409 INFO [RS:3;jenkins-hbase4:42043] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:22,410 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:55:22,414 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:22,415 INFO [RS:2;jenkins-hbase4:43495] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:22,416 INFO [RS:3;jenkins-hbase4:42043] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:22,417 INFO [RS:1;jenkins-hbase4:34613] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:22,418 INFO [RS:3;jenkins-hbase4:42043] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:22,418 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,418 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,422 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:22,422 INFO [RS:2;jenkins-hbase4:43495] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:22,422 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,423 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,423 INFO [RS:1;jenkins-hbase4:34613] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:22,423 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,423 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:22,424 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,423 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,424 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64522908, jitterRate=-0.03853374719619751}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:55:22,424 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,424 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:55:22,424 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,424 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:55:22,424 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:22,424 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:22,426 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,425 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:22,424 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:55:22,427 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:55:22,427 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:55:22,427 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:55:22,427 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,427 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,428 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,428 DEBUG [RS:0;jenkins-hbase4:45061] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,430 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:55:22,431 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,430 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,432 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,432 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,431 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,431 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:55:22,432 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,432 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,433 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,432 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,433 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:22,433 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:2;jenkins-hbase4:43495] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:22,434 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,433 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:1;jenkins-hbase4:34613] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,434 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:22,435 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,435 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,435 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,435 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:22,435 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-19 05:55:22,435 DEBUG [RS:3;jenkins-hbase4:42043] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:22,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-19 05:55:22,448 INFO [RS:0;jenkins-hbase4:45061] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:22,449 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45061,1689746121819-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,450 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,450 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-19 05:55:22,450 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,450 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,450 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,458 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,458 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,459 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,459 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,459 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,462 INFO [RS:3;jenkins-hbase4:42043] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:22,462 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42043,1689746121940-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,463 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-19 05:55:22,465 INFO [RS:0;jenkins-hbase4:45061] regionserver.Replication(203): jenkins-hbase4.apache.org,45061,1689746121819 started 2023-07-19 05:55:22,465 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45061,1689746121819, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45061, sessionid=0x1017c2601c00008 2023-07-19 05:55:22,466 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:22,466 DEBUG [RS:0;jenkins-hbase4:45061] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,466 DEBUG [RS:0;jenkins-hbase4:45061] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45061,1689746121819' 2023-07-19 05:55:22,466 DEBUG [RS:0;jenkins-hbase4:45061] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-19 05:55:22,466 DEBUG [RS:0;jenkins-hbase4:45061] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-19 05:55:22,468 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:22,468 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:22,468 DEBUG [RS:0;jenkins-hbase4:45061] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:22,468 DEBUG [RS:0;jenkins-hbase4:45061] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45061,1689746121819' 2023-07-19 05:55:22,468 DEBUG [RS:0;jenkins-hbase4:45061] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-19 05:55:22,468 DEBUG [RS:0;jenkins-hbase4:45061] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-19 05:55:22,469 DEBUG [RS:0;jenkins-hbase4:45061] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:22,469 INFO [RS:0;jenkins-hbase4:45061] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:22,469 INFO [RS:0;jenkins-hbase4:45061] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:22,471 INFO [RS:2;jenkins-hbase4:43495] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:22,471 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43495,1689746121886-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,472 INFO [RS:1;jenkins-hbase4:34613] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:22,472 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34613,1689746121857-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,481 INFO [RS:3;jenkins-hbase4:42043] regionserver.Replication(203): jenkins-hbase4.apache.org,42043,1689746121940 started 2023-07-19 05:55:22,481 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42043,1689746121940, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42043, sessionid=0x1017c2601c0000b 2023-07-19 05:55:22,481 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:22,481 DEBUG [RS:3;jenkins-hbase4:42043] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,481 DEBUG [RS:3;jenkins-hbase4:42043] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42043,1689746121940' 2023-07-19 05:55:22,481 DEBUG [RS:3;jenkins-hbase4:42043] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-19 05:55:22,481 INFO [RS:2;jenkins-hbase4:43495] regionserver.Replication(203): jenkins-hbase4.apache.org,43495,1689746121886 started 2023-07-19 05:55:22,481 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43495,1689746121886, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43495, sessionid=0x1017c2601c0000a 2023-07-19 05:55:22,481 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:22,481 DEBUG [RS:2;jenkins-hbase4:43495] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,481 DEBUG [RS:2;jenkins-hbase4:43495] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43495,1689746121886' 2023-07-19 05:55:22,482 DEBUG [RS:2;jenkins-hbase4:43495] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-19 05:55:22,482 DEBUG [RS:3;jenkins-hbase4:42043] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-19 05:55:22,482 DEBUG [RS:2;jenkins-hbase4:43495] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-19 05:55:22,482 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:22,482 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:22,482 DEBUG [RS:3;jenkins-hbase4:42043] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,483 DEBUG [RS:3;jenkins-hbase4:42043] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42043,1689746121940' 2023-07-19 05:55:22,483 DEBUG [RS:3;jenkins-hbase4:42043] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-19 05:55:22,483 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:22,483 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:22,483 DEBUG [RS:2;jenkins-hbase4:43495] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:22,483 DEBUG [RS:2;jenkins-hbase4:43495] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43495,1689746121886' 2023-07-19 05:55:22,483 DEBUG [RS:2;jenkins-hbase4:43495] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-19 05:55:22,483 INFO [RS:1;jenkins-hbase4:34613] regionserver.Replication(203): jenkins-hbase4.apache.org,34613,1689746121857 started 2023-07-19 05:55:22,483 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34613,1689746121857, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34613, sessionid=0x1017c2601c00009 2023-07-19 05:55:22,483 DEBUG [RS:2;jenkins-hbase4:43495] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-19 05:55:22,483 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:22,484 DEBUG [RS:3;jenkins-hbase4:42043] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-19 05:55:22,484 DEBUG [RS:1;jenkins-hbase4:34613] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,484 DEBUG [RS:1;jenkins-hbase4:34613] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34613,1689746121857' 2023-07-19 05:55:22,484 DEBUG [RS:1;jenkins-hbase4:34613] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-19 05:55:22,484 DEBUG [RS:2;jenkins-hbase4:43495] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:22,484 DEBUG [RS:3;jenkins-hbase4:42043] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:22,484 INFO [RS:3;jenkins-hbase4:42043] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:22,484 INFO [RS:2;jenkins-hbase4:43495] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:22,484 INFO [RS:2;jenkins-hbase4:43495] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:22,484 DEBUG [RS:1;jenkins-hbase4:34613] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-19 05:55:22,484 INFO [RS:3;jenkins-hbase4:42043] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:22,485 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:22,485 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:22,485 DEBUG [RS:1;jenkins-hbase4:34613] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:22,485 DEBUG [RS:1;jenkins-hbase4:34613] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34613,1689746121857' 2023-07-19 05:55:22,485 DEBUG [RS:1;jenkins-hbase4:34613] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-19 05:55:22,485 DEBUG [RS:1;jenkins-hbase4:34613] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-19 05:55:22,486 DEBUG [RS:1;jenkins-hbase4:34613] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:22,486 INFO [RS:1;jenkins-hbase4:34613] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:22,486 INFO [RS:1;jenkins-hbase4:34613] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:22,569 INFO [RS:0;jenkins-hbase4:45061] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:22,572 INFO [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45061%2C1689746121819.jenkins-hbase4.apache.org%2C45061%2C1689746121819.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:22,585 INFO [RS:3;jenkins-hbase4:42043] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:22,585 INFO [RS:2;jenkins-hbase4:43495] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:22,586 INFO [RS:1;jenkins-hbase4:34613] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:22,591 INFO [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42043%2C1689746121940.jenkins-hbase4.apache.org%2C42043%2C1689746121940.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:22,592 INFO [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43495%2C1689746121886.jenkins-hbase4.apache.org%2C43495%2C1689746121886.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:22,593 INFO [RS:1;jenkins-hbase4:34613] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34613%2C1689746121857.jenkins-hbase4.apache.org%2C34613%2C1689746121857.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:22,595 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:22,597 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:22,598 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:22,607 INFO [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819/jenkins-hbase4.apache.org%2C45061%2C1689746121819.jenkins-hbase4.apache.org%2C45061%2C1689746121819.regiongroup-0.1689746122573 2023-07-19 05:55:22,607 DEBUG [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK]] 2023-07-19 05:55:22,613 DEBUG [jenkins-hbase4:36183] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-19 05:55:22,613 DEBUG [jenkins-hbase4:36183] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:22,619 DEBUG [jenkins-hbase4:36183] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:22,619 DEBUG [jenkins-hbase4:36183] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:22,619 DEBUG [jenkins-hbase4:36183] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-19 05:55:22,619 DEBUG [jenkins-hbase4:36183] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-19 05:55:22,619 DEBUG [jenkins-hbase4:36183] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:22,621 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42043,1689746121940, state=OPENING 2023-07-19 05:55:22,622 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:22,623 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:22,623 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:22,624 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-19 05:55:22,632 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:22,632 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42043,1689746121940}] 2023-07-19 05:55:22,632 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-19 05:55:22,641 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:22,644 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:22,644 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:22,644 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:22,645 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:22,645 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:22,648 INFO [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940/jenkins-hbase4.apache.org%2C42043%2C1689746121940.jenkins-hbase4.apache.org%2C42043%2C1689746121940.regiongroup-0.1689746122596 2023-07-19 05:55:22,657 DEBUG [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK]] 2023-07-19 05:55:22,658 INFO [RS:1;jenkins-hbase4:34613] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857/jenkins-hbase4.apache.org%2C34613%2C1689746121857.jenkins-hbase4.apache.org%2C34613%2C1689746121857.regiongroup-0.1689746122601 2023-07-19 05:55:22,658 INFO [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886/jenkins-hbase4.apache.org%2C43495%2C1689746121886.jenkins-hbase4.apache.org%2C43495%2C1689746121886.regiongroup-0.1689746122601 2023-07-19 05:55:22,658 DEBUG [RS:1;jenkins-hbase4:34613] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK]] 2023-07-19 05:55:22,659 DEBUG [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK]] 2023-07-19 05:55:22,811 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:22,811 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:22,812 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55500, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:22,821 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-19 05:55:22,821 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:22,821 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:22,821 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:22,823 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42043%2C1689746121940.meta, suffix=.meta, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:22,838 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:22,839 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:22,839 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:22,848 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940/jenkins-hbase4.apache.org%2C42043%2C1689746121940.meta.1689746122824.meta 2023-07-19 05:55:22,850 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK]] 2023-07-19 05:55:22,850 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:22,851 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-19 05:55:22,851 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-19 05:55:22,854 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-19 05:55:22,854 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-19 05:55:22,854 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:22,854 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-19 05:55:22,854 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-19 05:55:22,856 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:55:22,857 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:55:22,857 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:55:22,858 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:55:22,859 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,859 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:55:22,860 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:22,860 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:22,861 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:55:22,866 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,866 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:55:22,868 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:55:22,868 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:55:22,868 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:55:22,869 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:22,871 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:55:22,874 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:55:22,877 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:55:22,879 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:55:22,881 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74247262, jitterRate=0.10637041926383972}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:55:22,881 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:55:22,885 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689746122811 2023-07-19 05:55:22,891 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-19 05:55:22,892 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-19 05:55:22,893 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42043,1689746121940, state=OPEN 2023-07-19 05:55:22,895 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-19 05:55:22,895 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-19 05:55:22,898 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-19 05:55:22,898 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42043,1689746121940 in 263 msec 2023-07-19 05:55:22,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-19 05:55:22,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 463 msec 2023-07-19 05:55:22,905 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 660 msec 2023-07-19 05:55:22,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689746122909, completionTime=-1 2023-07-19 05:55:22,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-19 05:55:22,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-19 05:55:22,913 DEBUG [hconnection-0x2ba5675e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:22,915 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55516, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:22,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-19 05:55:22,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689746182917 2023-07-19 05:55:22,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689746242917 2023-07-19 05:55:22,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 7 msec 2023-07-19 05:55:22,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36183,1689746121767-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36183,1689746121767-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36183,1689746121767-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36183, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:22,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-19 05:55:22,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:22,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-19 05:55:22,932 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-19 05:55:22,934 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-19 05:55:22,935 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-19 05:55:22,941 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:22,942 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a empty. 2023-07-19 05:55:22,943 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:22,943 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-19 05:55:23,024 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:23,026 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => ecfca84342d05307f433bda44290dd5a, NAME => 'hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp 2023-07-19 05:55:23,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:23,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing ecfca84342d05307f433bda44290dd5a, disabling compactions & flushes 2023-07-19 05:55:23,043 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. after waiting 0 ms 2023-07-19 05:55:23,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,043 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:55:23,047 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-19 05:55:23,050 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746123049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689746123049"}]},"ts":"1689746123049"} 2023-07-19 05:55:23,054 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-19 05:55:23,055 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-19 05:55:23,055 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746123055"}]},"ts":"1689746123055"} 2023-07-19 05:55:23,058 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-19 05:55:23,062 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:23,063 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:23,063 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:23,063 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-19 05:55:23,063 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-19 05:55:23,063 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:23,063 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN}] 2023-07-19 05:55:23,067 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN 2023-07-19 05:55:23,068 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42043,1689746121940; forceNewPlan=false, retain=false 2023-07-19 05:55:23,218 INFO [jenkins-hbase4:36183] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-19 05:55:23,220 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ecfca84342d05307f433bda44290dd5a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:23,221 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746123220"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746123220"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746123220"}]},"ts":"1689746123220"} 2023-07-19 05:55:23,224 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ecfca84342d05307f433bda44290dd5a, server=jenkins-hbase4.apache.org,42043,1689746121940}] 2023-07-19 05:55:23,384 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,384 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:23,387 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42043%2C1689746121940.jenkins-hbase4.apache.org%2C42043%2C1689746121940.regiongroup-1, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:23,402 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:23,403 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:23,404 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:23,410 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940/jenkins-hbase4.apache.org%2C42043%2C1689746121940.jenkins-hbase4.apache.org%2C42043%2C1689746121940.regiongroup-1.1689746123387 2023-07-19 05:55:23,410 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK]] 2023-07-19 05:55:23,411 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ecfca84342d05307f433bda44290dd5a, NAME => 'hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:23,411 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,411 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:23,411 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,411 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,413 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,416 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info 2023-07-19 05:55:23,416 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info 2023-07-19 05:55:23,417 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ecfca84342d05307f433bda44290dd5a columnFamilyName info 2023-07-19 05:55:23,419 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(310): Store=ecfca84342d05307f433bda44290dd5a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:23,421 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,422 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,426 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:23,429 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:23,429 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ecfca84342d05307f433bda44290dd5a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60896997, jitterRate=-0.09256403148174286}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:23,430 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:55:23,431 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., pid=6, masterSystemTime=1689746123377 2023-07-19 05:55:23,435 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,435 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:23,436 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ecfca84342d05307f433bda44290dd5a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:23,436 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746123436"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746123436"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746123436"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746123436"}]},"ts":"1689746123436"} 2023-07-19 05:55:23,442 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-19 05:55:23,442 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ecfca84342d05307f433bda44290dd5a, server=jenkins-hbase4.apache.org,42043,1689746121940 in 215 msec 2023-07-19 05:55:23,445 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-19 05:55:23,446 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN in 379 msec 2023-07-19 05:55:23,447 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-19 05:55:23,447 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746123447"}]},"ts":"1689746123447"} 2023-07-19 05:55:23,449 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-19 05:55:23,452 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-19 05:55:23,454 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 523 msec 2023-07-19 05:55:23,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-19 05:55:23,540 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-19 05:55:23,540 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:23,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-19 05:55:23,558 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-19 05:55:23,562 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 17 msec 2023-07-19 05:55:23,569 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-19 05:55:23,591 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-19 05:55:23,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 26 msec 2023-07-19 05:55:23,605 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-19 05:55:23,608 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-19 05:55:23,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.621sec 2023-07-19 05:55:23,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-19 05:55:23,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-19 05:55:23,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-19 05:55:23,609 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36183,1689746121767-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-19 05:55:23,609 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36183,1689746121767-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-19 05:55:23,612 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-19 05:55:23,678 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x6c62f4e9 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:23,684 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c81ccf0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:23,686 DEBUG [hconnection-0x37a0b0ce-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:23,688 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55530, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:23,691 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:23,703 DEBUG [Listener at localhost/45003] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-19 05:55:23,704 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35318, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-19 05:55:23,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:23,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-19 05:55:23,724 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-19 05:55:23,726 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-19 05:55:23,728 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:23,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-19 05:55:23,729 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 empty. 2023-07-19 05:55:23,730 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:23,730 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-19 05:55:23,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:23,756 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:23,757 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8c7505042d76fb8c47639ac419e6a996, NAME => 't1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp 2023-07-19 05:55:23,770 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:23,771 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 8c7505042d76fb8c47639ac419e6a996, disabling compactions & flushes 2023-07-19 05:55:23,771 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:23,771 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:23,771 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. after waiting 0 ms 2023-07-19 05:55:23,771 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:23,771 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:23,771 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 8c7505042d76fb8c47639ac419e6a996: 2023-07-19 05:55:23,774 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-19 05:55:23,776 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746123776"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689746123776"}]},"ts":"1689746123776"} 2023-07-19 05:55:23,779 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-19 05:55:23,780 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-19 05:55:23,780 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746123780"}]},"ts":"1689746123780"} 2023-07-19 05:55:23,782 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-19 05:55:23,787 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:23,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:23,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:23,788 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:23,788 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN}] 2023-07-19 05:55:23,790 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN 2023-07-19 05:55:23,792 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43759,1689746117439; forceNewPlan=false, retain=false 2023-07-19 05:55:23,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:23,874 INFO [jenkins-hbase4:43215Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:23,875 INFO [jenkins-hbase4:43759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:23,942 INFO [jenkins-hbase4:39519] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-19 05:55:23,944 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=8c7505042d76fb8c47639ac419e6a996, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:23,944 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746123943"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746123943"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746123943"}]},"ts":"1689746123943"} 2023-07-19 05:55:23,947 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 8c7505042d76fb8c47639ac419e6a996, server=jenkins-hbase4.apache.org,43759,1689746117439}] 2023-07-19 05:55:24,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:24,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:24,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:24,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:24,128 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:24,128 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:24,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109 2023-07-19 05:55:24,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:55:24,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8c7505042d76fb8c47639ac419e6a996, NAME => 't1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:24,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:24,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,137 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,139 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1 2023-07-19 05:55:24,139 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1 2023-07-19 05:55:24,140 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c7505042d76fb8c47639ac419e6a996 columnFamilyName cf1 2023-07-19 05:55:24,141 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(310): Store=8c7505042d76fb8c47639ac419e6a996/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:24,141 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,143 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep 2023-07-19 05:55:24,143 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep 2023-07-19 05:55:24,143 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c7505042d76fb8c47639ac419e6a996 columnFamilyName norep 2023-07-19 05:55:24,144 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(310): Store=8c7505042d76fb8c47639ac419e6a996/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:24,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:55:24,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:24,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:24,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8c7505042d76fb8c47639ac419e6a996; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69734892, jitterRate=0.03913086652755737}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:55:24,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8c7505042d76fb8c47639ac419e6a996: 2023-07-19 05:55:24,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996., pid=11, masterSystemTime=1689746124099 2023-07-19 05:55:24,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:24,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:24,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=8c7505042d76fb8c47639ac419e6a996, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:24,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746124159"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746124159"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746124159"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746124159"}]},"ts":"1689746124159"} 2023-07-19 05:55:24,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-19 05:55:24,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 8c7505042d76fb8c47639ac419e6a996, server=jenkins-hbase4.apache.org,43759,1689746117439 in 216 msec 2023-07-19 05:55:24,168 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-19 05:55:24,169 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN in 377 msec 2023-07-19 05:55:24,170 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-19 05:55:24,170 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746124170"}]},"ts":"1689746124170"} 2023-07-19 05:55:24,172 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-19 05:55:24,175 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-19 05:55:24,177 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 456 msec 2023-07-19 05:55:24,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:24,349 INFO [Listener at localhost/45003] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-19 05:55:24,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:24,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-19 05:55:24,355 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-19 05:55:24,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-19 05:55:24,358 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-19 05:55:24,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:24,360 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,361 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 empty. 2023-07-19 05:55:24,361 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,361 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-19 05:55:24,385 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:24,387 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8f71d242d3ffe63977903bdc0b5b9fe2, NAME => 't2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.tmp 2023-07-19 05:55:24,415 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:24,415 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 8f71d242d3ffe63977903bdc0b5b9fe2, disabling compactions & flushes 2023-07-19 05:55:24,415 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,415 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,415 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. after waiting 0 ms 2023-07-19 05:55:24,415 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,415 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,415 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 8f71d242d3ffe63977903bdc0b5b9fe2: 2023-07-19 05:55:24,420 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-19 05:55:24,424 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746124423"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689746124423"}]},"ts":"1689746124423"} 2023-07-19 05:55:24,426 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-19 05:55:24,428 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-19 05:55:24,428 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746124428"}]},"ts":"1689746124428"} 2023-07-19 05:55:24,430 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-19 05:55:24,435 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:24,437 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:24,437 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:24,437 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:24,437 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN}] 2023-07-19 05:55:24,440 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN 2023-07-19 05:55:24,441 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43215,1689746117482; forceNewPlan=false, retain=false 2023-07-19 05:55:24,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:24,591 INFO [jenkins-hbase4:39519] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-19 05:55:24,593 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=8f71d242d3ffe63977903bdc0b5b9fe2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:24,593 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746124593"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746124593"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746124593"}]},"ts":"1689746124593"} 2023-07-19 05:55:24,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 8f71d242d3ffe63977903bdc0b5b9fe2, server=jenkins-hbase4.apache.org,43215,1689746117482}] 2023-07-19 05:55:24,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:24,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8f71d242d3ffe63977903bdc0b5b9fe2, NAME => 't2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:24,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:24,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,758 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,760 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1 2023-07-19 05:55:24,760 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1 2023-07-19 05:55:24,760 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f71d242d3ffe63977903bdc0b5b9fe2 columnFamilyName cf1 2023-07-19 05:55:24,761 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(310): Store=8f71d242d3ffe63977903bdc0b5b9fe2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:24,761 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,763 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep 2023-07-19 05:55:24,763 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep 2023-07-19 05:55:24,763 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f71d242d3ffe63977903bdc0b5b9fe2 columnFamilyName norep 2023-07-19 05:55:24,764 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(310): Store=8f71d242d3ffe63977903bdc0b5b9fe2/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:24,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:55:24,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:24,772 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:55:24,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:24,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8f71d242d3ffe63977903bdc0b5b9fe2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61341358, jitterRate=-0.08594253659248352}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:55:24,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8f71d242d3ffe63977903bdc0b5b9fe2: 2023-07-19 05:55:24,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2., pid=14, masterSystemTime=1689746124748 2023-07-19 05:55:24,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:24,778 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=8f71d242d3ffe63977903bdc0b5b9fe2, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:24,778 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746124778"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746124778"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746124778"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746124778"}]},"ts":"1689746124778"} 2023-07-19 05:55:24,799 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-19 05:55:24,799 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 8f71d242d3ffe63977903bdc0b5b9fe2, server=jenkins-hbase4.apache.org,43215,1689746117482 in 184 msec 2023-07-19 05:55:24,805 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-19 05:55:24,806 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN in 362 msec 2023-07-19 05:55:24,807 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-19 05:55:24,808 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746124808"}]},"ts":"1689746124808"} 2023-07-19 05:55:24,815 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-19 05:55:24,823 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-19 05:55:24,835 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 472 msec 2023-07-19 05:55:24,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:24,964 INFO [Listener at localhost/45003] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-19 05:55:24,967 DEBUG [Listener at localhost/45003] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-19 05:55:24,969 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59784, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-19 05:55:24,969 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-19 05:55:24,970 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-19 05:55:24,971 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-19 05:55:24,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:24,973 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-19 05:55:24,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-19 05:55:24,981 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-19 05:55:24,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-19 05:55:24,983 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-19 05:55:24,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:24,986 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:24,988 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 empty. 2023-07-19 05:55:24,990 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:24,990 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-19 05:55:25,033 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:25,035 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2054d9ec275367d29389fbedb9c09867, NAME => 't1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp 2023-07-19 05:55:25,068 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:25,069 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 2054d9ec275367d29389fbedb9c09867, disabling compactions & flushes 2023-07-19 05:55:25,069 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,069 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,069 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. after waiting 0 ms 2023-07-19 05:55:25,069 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,069 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,069 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:55:25,076 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-19 05:55:25,077 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746125077"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689746125077"}]},"ts":"1689746125077"} 2023-07-19 05:55:25,080 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-19 05:55:25,081 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-19 05:55:25,081 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746125081"}]},"ts":"1689746125081"} 2023-07-19 05:55:25,083 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-19 05:55:25,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:25,088 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:25,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:25,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:25,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-19 05:55:25,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-19 05:55:25,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:25,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN}] 2023-07-19 05:55:25,093 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN 2023-07-19 05:55:25,095 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45061,1689746121819; forceNewPlan=false, retain=false 2023-07-19 05:55:25,245 INFO [jenkins-hbase4:36183] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-19 05:55:25,247 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=2054d9ec275367d29389fbedb9c09867, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:25,247 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746125246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746125246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746125246"}]},"ts":"1689746125246"} 2023-07-19 05:55:25,250 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 2054d9ec275367d29389fbedb9c09867, server=jenkins-hbase4.apache.org,45061,1689746121819}] 2023-07-19 05:55:25,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:25,404 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:25,405 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:25,406 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53456, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:25,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:25,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45061%2C1689746121819.jenkins-hbase4.apache.org%2C45061%2C1689746121819.regiongroup-1, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:25,431 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:25,431 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:25,431 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:25,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819/jenkins-hbase4.apache.org%2C45061%2C1689746121819.jenkins-hbase4.apache.org%2C45061%2C1689746121819.regiongroup-1.1689746125415 2023-07-19 05:55:25,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK]] 2023-07-19 05:55:25,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2054d9ec275367d29389fbedb9c09867, NAME => 't1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:25,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:25,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,471 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,473 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1 2023-07-19 05:55:25,474 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1 2023-07-19 05:55:25,475 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2054d9ec275367d29389fbedb9c09867 columnFamilyName cf1 2023-07-19 05:55:25,477 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(310): Store=2054d9ec275367d29389fbedb9c09867/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:25,477 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,479 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/norep 2023-07-19 05:55:25,479 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/norep 2023-07-19 05:55:25,480 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2054d9ec275367d29389fbedb9c09867 columnFamilyName norep 2023-07-19 05:55:25,481 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(310): Store=2054d9ec275367d29389fbedb9c09867/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:25,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:55:25,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:25,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:25,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2054d9ec275367d29389fbedb9c09867; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70809826, jitterRate=0.05514863133430481}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:55:25,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:55:25,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867., pid=11, masterSystemTime=1689746125404 2023-07-19 05:55:25,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:25,506 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=2054d9ec275367d29389fbedb9c09867, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:25,506 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746125506"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746125506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746125506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746125506"}]},"ts":"1689746125506"} 2023-07-19 05:55:25,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-19 05:55:25,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 2054d9ec275367d29389fbedb9c09867, server=jenkins-hbase4.apache.org,45061,1689746121819 in 259 msec 2023-07-19 05:55:25,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-19 05:55:25,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN in 422 msec 2023-07-19 05:55:25,516 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-19 05:55:25,517 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746125516"}]},"ts":"1689746125516"} 2023-07-19 05:55:25,519 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-19 05:55:25,523 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-19 05:55:25,526 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 551 msec 2023-07-19 05:55:25,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-19 05:55:25,589 INFO [Listener at localhost/45003] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-19 05:55:25,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:25,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-19 05:55:25,597 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-19 05:55:25,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-19 05:55:25,598 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-19 05:55:25,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:25,600 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:25,601 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b empty. 2023-07-19 05:55:25,602 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:25,602 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-19 05:55:25,648 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-19 05:55:25,651 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 168d6c8e660cdb03a471b9cc5e7a3e2b, NAME => 't2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.tmp 2023-07-19 05:55:25,678 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:25,678 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 168d6c8e660cdb03a471b9cc5e7a3e2b, disabling compactions & flushes 2023-07-19 05:55:25,678 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:25,678 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:25,678 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. after waiting 0 ms 2023-07-19 05:55:25,678 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:25,679 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:25,679 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:55:25,683 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-19 05:55:25,684 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746125684"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689746125684"}]},"ts":"1689746125684"} 2023-07-19 05:55:25,692 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-19 05:55:25,693 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-19 05:55:25,694 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746125693"}]},"ts":"1689746125693"} 2023-07-19 05:55:25,696 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-19 05:55:25,702 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-19 05:55:25,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:25,704 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-19 05:55:25,704 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-19 05:55:25,704 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-19 05:55:25,704 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-19 05:55:25,704 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-19 05:55:25,704 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN}] 2023-07-19 05:55:25,709 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN 2023-07-19 05:55:25,711 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43495,1689746121886; forceNewPlan=false, retain=false 2023-07-19 05:55:25,862 INFO [jenkins-hbase4:36183] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-19 05:55:25,864 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:25,864 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746125864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746125864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746125864"}]},"ts":"1689746125864"} 2023-07-19 05:55:25,866 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 168d6c8e660cdb03a471b9cc5e7a3e2b, server=jenkins-hbase4.apache.org,43495,1689746121886}] 2023-07-19 05:55:25,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:26,022 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:26,022 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:26,024 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33046, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:26,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:26,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:26,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43495%2C1689746121886.jenkins-hbase4.apache.org%2C43495%2C1689746121886.regiongroup-1, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:26,046 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:26,046 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:26,046 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:26,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886/jenkins-hbase4.apache.org%2C43495%2C1689746121886.jenkins-hbase4.apache.org%2C43495%2C1689746121886.regiongroup-1.1689746126031 2023-07-19 05:55:26,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK]] 2023-07-19 05:55:26,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 168d6c8e660cdb03a471b9cc5e7a3e2b, NAME => 't2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:26,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:26,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,065 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,067 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1 2023-07-19 05:55:26,067 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1 2023-07-19 05:55:26,067 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 168d6c8e660cdb03a471b9cc5e7a3e2b columnFamilyName cf1 2023-07-19 05:55:26,068 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(310): Store=168d6c8e660cdb03a471b9cc5e7a3e2b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:26,068 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,070 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/norep 2023-07-19 05:55:26,070 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/norep 2023-07-19 05:55:26,070 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 168d6c8e660cdb03a471b9cc5e7a3e2b columnFamilyName norep 2023-07-19 05:55:26,071 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(310): Store=168d6c8e660cdb03a471b9cc5e7a3e2b/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:26,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:55:26,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:26,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-19 05:55:26,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 168d6c8e660cdb03a471b9cc5e7a3e2b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60994106, jitterRate=-0.0911169946193695}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:55:26,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:55:26,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b., pid=14, masterSystemTime=1689746126022 2023-07-19 05:55:26,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:26,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:26,085 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:26,085 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746126085"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746126085"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746126085"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746126085"}]},"ts":"1689746126085"} 2023-07-19 05:55:26,091 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-19 05:55:26,091 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 168d6c8e660cdb03a471b9cc5e7a3e2b, server=jenkins-hbase4.apache.org,43495,1689746121886 in 222 msec 2023-07-19 05:55:26,093 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-19 05:55:26,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN in 387 msec 2023-07-19 05:55:26,095 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-19 05:55:26,095 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689746126095"}]},"ts":"1689746126095"} 2023-07-19 05:55:26,097 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-19 05:55:26,101 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-19 05:55:26,104 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 510 msec 2023-07-19 05:55:26,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-19 05:55:26,206 INFO [Listener at localhost/45003] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-19 05:55:26,207 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x22818760 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:26,216 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b79953, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:26,216 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x23ab8a15 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:26,232 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@dbfb387, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:26,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:65243:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-19 05:55:26,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-19 05:55:26,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-19 05:55:26,273 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-19 05:55:26,273 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@74940f62check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:26,279 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@74940f62check-peer-cluster-id0x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:26,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@74940f62check-peer-cluster-id-0x1017c2601c00014 connected 2023-07-19 05:55:26,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-19 05:55:26,383 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@74940f62check-peer-cluster-id-0x1017c2601c00014, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:26,392 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:26,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-19 05:55:26,445 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-19 05:55:26,446 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-19 05:55:26,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-19 05:55:26,576 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43215] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-19 05:55:26,576 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43759] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-19 05:55:26,578 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-19 05:55:26,578 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-19 05:55:26,590 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-19 05:55:26,590 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-19 05:55:26,609 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:55:26,609 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:55:26,638 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-19 05:55:26,638 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-19 05:55:26,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39519] master.HMaster(3641): Remote procedure done, pid=17 2023-07-19 05:55:26,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.HMaster(3641): Remote procedure done, pid=16 2023-07-19 05:55:26,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39519] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,43759,1689746117439 suceeded 2023-07-19 05:55:26,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,43215,1689746117482 suceeded 2023-07-19 05:55:26,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 237 msec 2023-07-19 05:55:26,664 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-19 05:55:26,664 INFO [PEWorker-3] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:65243:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-19 05:55:26,664 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 237 msec 2023-07-19 05:55:26,666 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 417 msec 2023-07-19 05:55:26,675 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1a77650e to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:26,675 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x562919b5 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:26,685 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6bdba1a7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:26,685 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3ad4a57, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:26,688 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:26,688 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:26,693 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:26,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c00017 connected 2023-07-19 05:55:26,694 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:26,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c00018 connected 2023-07-19 05:55:26,695 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:26,696 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:26,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1 2023-07-19 05:55:26,698 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1 2023-07-19 05:55:26,703 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:26,703 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:26,704 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0 2023-07-19 05:55:26,704 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0 2023-07-19 05:55:26,704 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1 2023-07-19 05:55:26,704 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:26,704 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:26,704 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1 2023-07-19 05:55:26,705 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0 2023-07-19 05:55:26,705 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0 2023-07-19 05:55:26,786 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:26,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-19 05:55:26,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-19 05:55:26,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:26,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:26,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-19 05:55:26,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:26,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:26,792 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:26,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:26,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:26,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-19 05:55:26,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:26,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39519] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-19 05:55:26,865 DEBUG [Listener at localhost/45003] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-19 05:55:26,868 DEBUG [hconnection-0x55f51523-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:26,869 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40888, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:26,897 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:26,906 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:26,906 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-19 05:55:26,907 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1414 2023-07-19 05:55:26,907 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 10 WAL entries eligible for replication 2023-07-19 05:55:26,907 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1414 2023-07-19 05:55:26,908 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/4=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/5=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/6=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/7=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/8=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/9=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/10=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/11=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/12=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/13=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109, lastWalPosition=1414, nbRowKeys=10, nbHFiles=0, heapSize=2080, lastSeqIds={}, endOfFile=false,usedBufferSize=2080] 2023-07-19 05:55:26,923 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:26,927 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52414, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-19 05:55:26,930 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:26,931 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] zookeeper.ReadOnlyZKClient(139): Connect 0x54eff967 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:26,938 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@619777ce, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:26,948 DEBUG [hconnection-0x5efc4de-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:26,949 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55534, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:26,984 DEBUG [hconnection-0x5efc4de-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:26,992 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53472, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,016 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,026 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-19 05:55:27,056 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 7084 2023-07-19 05:55:27,056 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 54 WAL entries eligible for replication 2023-07-19 05:55:27,056 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 7084 2023-07-19 05:55:27,057 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,061 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,066 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/14=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/15=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/16=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/17=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/18=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/19=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/20=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/21=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/22=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/23=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/24=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/25=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/26=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/27=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/28=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/29=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/30=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/31=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/32=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/33=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/34=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/35=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/36=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/37=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/38=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/39=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/40=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/41=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/42=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/43=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/44=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/45=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/46=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/47=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/48=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/49=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/50=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/51=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/52=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/53=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/54=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/55=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/56=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/57=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/58=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/59=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/60=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/61=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/62=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/63=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/64=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/65=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/66=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/67=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109, lastWalPosition=7084, nbRowKeys=54, nbHFiles=0, heapSize=11232, lastSeqIds={}, endOfFile=false,usedBufferSize=11232] 2023-07-19 05:55:27,074 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,085 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,112 DEBUG [Listener at localhost/45003] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,114 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,124 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,136 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1311 2023-07-19 05:55:27,136 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 9 WAL entries eligible for replication 2023-07-19 05:55:27,136 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1311 2023-07-19 05:55:27,136 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/4=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/5=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/6=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/7=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/8=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/9=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/10=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/11=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/12=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=1311, nbRowKeys=9, nbHFiles=0, heapSize=1872, lastSeqIds={}, endOfFile=false,usedBufferSize=1872] 2023-07-19 05:55:27,140 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:27,141 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52418, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-19 05:55:27,142 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,146 DEBUG [hconnection-0x5efc4de-shared-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,148 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33062, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,158 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34613] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,171 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,177 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,177 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 36 WAL entries eligible for replication 2023-07-19 05:55:27,177 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,177 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/68=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/69=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/70=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/71=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/72=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/73=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/74=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/75=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/76=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/77=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/78=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/79=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/80=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/81=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/82=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/83=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/84=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/85=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/86=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/87=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/88=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/89=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/90=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/91=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/92=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/93=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/94=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/95=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/96=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/97=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/98=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/99=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/100=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/101=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/102=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109, lastWalPosition=10975, nbRowKeys=36, nbHFiles=0, heapSize=7488, lastSeqIds={}, endOfFile=false,usedBufferSize=7488] 2023-07-19 05:55:27,178 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:27,180 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53484, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-19 05:55:27,182 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45061] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,183 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45061] zookeeper.ReadOnlyZKClient(139): Connect 0x3d184fa0 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:27,190 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45061] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@781ac667, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:27,192 DEBUG [hconnection-0x4e6aab10-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,193 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55538, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,201 DEBUG [hconnection-0x4e6aab10-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,203 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53496, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,207 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45061] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,245 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,264 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 6979 2023-07-19 05:55:27,264 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 54 WAL entries eligible for replication 2023-07-19 05:55:27,264 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/13=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/14=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/15=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/16=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/17=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/18=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/19=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/20=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/21=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/22=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/23=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/24=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/25=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/26=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/27=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/28=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/29=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/30=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/31=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/32=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/33=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/34=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/35=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/36=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/37=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/38=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/39=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/40=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/41=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/42=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/43=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/44=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/45=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/46=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/47=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/48=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/49=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/50=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/51=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/52=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/53=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/54=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/55=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/56=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/57=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/58=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/59=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/60=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/61=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/62=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/63=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/64=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/65=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/66=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=6979, nbRowKeys=54, nbHFiles=0, heapSize=11232, lastSeqIds={}, endOfFile=false,usedBufferSize=11232] 2023-07-19 05:55:27,265 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 6979 2023-07-19 05:55:27,271 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:27,273 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33072, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-19 05:55:27,275 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,276 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] zookeeper.ReadOnlyZKClient(139): Connect 0x0c59dda5 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:27,297 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,298 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@86658f6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:27,305 DEBUG [hconnection-0x4d64d79e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,314 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55548, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,315 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,325 DEBUG [hconnection-0x4d64d79e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,332 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33078, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,339 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,364 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,386 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11292 2023-07-19 05:55:27,386 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 41 WAL entries eligible for replication 2023-07-19 05:55:27,387 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/67=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/68=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/69=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/70=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/71=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/72=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/73=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/74=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/75=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/76=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/77=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/78=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/79=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/80=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/81=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/82=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/83=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/84=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/85=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/86=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/87=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/88=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/89=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/90=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/91=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/92=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/93=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/94=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/95=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/96=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/97=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/98=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/99=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/100=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/101=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/102=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/103=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/104=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/105=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/106=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/107=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=11292, nbRowKeys=41, nbHFiles=0, heapSize=8528, lastSeqIds={}, endOfFile=false,usedBufferSize=8528] 2023-07-19 05:55:27,387 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11292 2023-07-19 05:55:27,398 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,406 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,407 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,423 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,463 INFO [jenkins-hbase4:45061Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:27,464 INFO [jenkins-hbase4:45061Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 36 2023-07-19 05:55:27,473 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,480 INFO [jenkins-hbase4:42043Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:27,481 INFO [jenkins-hbase4:43495Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:27,481 INFO [jenkins-hbase4:43495Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 95 2023-07-19 05:55:27,483 INFO [jenkins-hbase4:34613Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:27,483 INFO [jenkins-hbase4:34613Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 73 2023-07-19 05:55:27,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 19156 2023-07-19 05:55:27,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 73 WAL entries eligible for replication 2023-07-19 05:55:27,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 19156 2023-07-19 05:55:27,510 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/108=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/109=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/110=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/111=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/112=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/113=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/114=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/115=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/116=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/117=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/118=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/119=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/120=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/121=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/122=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/123=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/124=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/125=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/126=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/127=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/128=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/129=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/130=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/131=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/132=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/133=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/134=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/135=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/136=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/137=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/138=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/139=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/140=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/141=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/142=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/143=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/144=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/145=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/146=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/147=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/148=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/149=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/150=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/151=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/152=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/153=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/154=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/155=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/156=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/157=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/158=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/159=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/160=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/161=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/162=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/163=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/164=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/165=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/166=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/167=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/168=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/169=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/170=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/171=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/172=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/173=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/174=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/175=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/176=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/177=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/178=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/179=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/180=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=19156, nbRowKeys=73, nbHFiles=0, heapSize=15184, lastSeqIds={}, endOfFile=false,usedBufferSize=15184] 2023-07-19 05:55:27,518 DEBUG [SinkThread-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:27,519 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,520 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55550, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-19 05:55:27,522 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42043] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,523 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42043] zookeeper.ReadOnlyZKClient(139): Connect 0x3ec0fb49 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:27,529 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42043] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@300702e1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:27,541 DEBUG [hconnection-0x34389039-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,542 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,542 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,549 DEBUG [hconnection-0x34389039-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:27,550 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33086, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:27,555 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42043] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,580 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,620 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:27,620 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 23 WAL entries eligible for replication 2023-07-19 05:55:27,620 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:27,620 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/181=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/182=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/183=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/184=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/185=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/186=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/187=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/188=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/189=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/190=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/191=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/192=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/193=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/194=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/195=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/196=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/197=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/198=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/199=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/200=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/201=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/202=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=21752, nbRowKeys=23, nbHFiles=0, heapSize=4784, lastSeqIds={}, endOfFile=false,usedBufferSize=4784] 2023-07-19 05:55:27,625 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:27,626 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,629 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43495] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:27,654 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,688 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,727 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:27,732 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,761 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,805 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,836 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:27,841 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:27,912 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:27,944 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:27,947 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:27,981 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,018 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,040 DEBUG [hconnection-0x32fdde62-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:28,042 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55566, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:28,048 DEBUG [hconnection-0x32fdde62-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:28,053 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53506, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:28,054 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,054 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,075 DEBUG [hconnection-0x32fdde62-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:28,077 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33102, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:28,083 DEBUG [Listener at localhost/45003] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-19 05:55:28,083 INFO [Listener at localhost/45003] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-19 05:55:28,084 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c62f4e9 to 127.0.0.1:65243 2023-07-19 05:55:28,084 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,085 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-19 05:55:28,085 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(257): Found active master hash=240875225, stopped=false 2023-07-19 05:55:28,085 INFO [Listener at localhost/45003] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:55:28,088 INFO [Listener at localhost/45003] procedure2.ProcedureExecutor(629): Stopping 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d93b016 to 127.0.0.1:65243 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:28,089 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45061,1689746121819' ***** 2023-07-19 05:55:28,088 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:55:28,089 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:55:28,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:28,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:28,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:28,091 INFO [RS:0;jenkins-hbase4:45061] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:55:28,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:28,091 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34613,1689746121857' ***** 2023-07-19 05:55:28,091 INFO [RS:0;jenkins-hbase4:45061] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:55:28,091 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:55:28,091 INFO [RS:0;jenkins-hbase4:45061] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:55:28,091 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:28,091 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:55:28,093 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(3305): Received CLOSE for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:28,093 INFO [RS:1;jenkins-hbase4:34613] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:55:28,093 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,093 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43495,1689746121886' ***** 2023-07-19 05:55:28,093 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:55:28,093 INFO [RS:1;jenkins-hbase4:34613] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:55:28,093 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:28,093 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:55:28,093 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28660803 to 127.0.0.1:65243 2023-07-19 05:55:28,095 INFO [RS:2;jenkins-hbase4:43495] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:55:28,093 INFO [RS:1;jenkins-hbase4:34613] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:55:28,096 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:55:28,096 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:28,093 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42043,1689746121940' ***** 2023-07-19 05:55:28,097 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29d5555b to 127.0.0.1:65243 2023-07-19 05:55:28,096 INFO [RS:2;jenkins-hbase4:43495] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:55:28,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2054d9ec275367d29389fbedb9c09867, disabling compactions & flushes 2023-07-19 05:55:28,095 DEBUG [RS:0;jenkins-hbase4:45061] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:28,097 INFO [RS:2;jenkins-hbase4:43495] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:55:28,097 DEBUG [RS:1;jenkins-hbase4:34613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,097 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(3305): Received CLOSE for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:28,097 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:55:28,097 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34613,1689746121857; all regions closed. 2023-07-19 05:55:28,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:28,097 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-19 05:55:28,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. after waiting 0 ms 2023-07-19 05:55:28,098 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1478): Online Regions={2054d9ec275367d29389fbedb9c09867=t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.} 2023-07-19 05:55:28,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:28,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2054d9ec275367d29389fbedb9c09867 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-19 05:55:28,099 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1504): Waiting on 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:28,102 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:28,103 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f58b9d2 to 127.0.0.1:65243 2023-07-19 05:55:28,103 DEBUG [RS:2;jenkins-hbase4:43495] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,103 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-19 05:55:28,103 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1478): Online Regions={168d6c8e660cdb03a471b9cc5e7a3e2b=t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.} 2023-07-19 05:55:28,103 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1504): Waiting on 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:28,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 168d6c8e660cdb03a471b9cc5e7a3e2b, disabling compactions & flushes 2023-07-19 05:55:28,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:28,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:28,108 INFO [RS:3;jenkins-hbase4:42043] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:55:28,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. after waiting 0 ms 2023-07-19 05:55:28,108 INFO [RS:3;jenkins-hbase4:42043] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:55:28,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:28,108 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:55:28,108 INFO [RS:3;jenkins-hbase4:42043] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:55:28,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 168d6c8e660cdb03a471b9cc5e7a3e2b 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-19 05:55:28,113 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(3305): Received CLOSE for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:28,113 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:28,113 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x00e2710f to 127.0.0.1:65243 2023-07-19 05:55:28,115 DEBUG [RS:3;jenkins-hbase4:42043] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,115 INFO [RS:3;jenkins-hbase4:42043] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:55:28,115 INFO [RS:3;jenkins-hbase4:42043] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:55:28,115 INFO [RS:3;jenkins-hbase4:42043] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:55:28,115 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-19 05:55:28,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ecfca84342d05307f433bda44290dd5a, disabling compactions & flushes 2023-07-19 05:55:28,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:28,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:28,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. after waiting 0 ms 2023-07-19 05:55:28,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:28,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ecfca84342d05307f433bda44290dd5a 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-19 05:55:28,124 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-19 05:55:28,124 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1478): Online Regions={ecfca84342d05307f433bda44290dd5a=hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., 1588230740=hbase:meta,,1.1588230740} 2023-07-19 05:55:28,126 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:55:28,126 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1504): Waiting on 1588230740, ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:28,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:55:28,126 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:55:28,126 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:55:28,126 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:55:28,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-19 05:55:28,138 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,143 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,151 DEBUG [RS:1;jenkins-hbase4:34613] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,151 INFO [RS:1;jenkins-hbase4:34613] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34613%2C1689746121857.jenkins-hbase4.apache.org%2C34613%2C1689746121857.regiongroup-0:(num 1689746122601) 2023-07-19 05:55:28,151 DEBUG [RS:1;jenkins-hbase4:34613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,151 INFO [RS:1;jenkins-hbase4:34613] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,154 INFO [RS:1;jenkins-hbase4:34613] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-19 05:55:28,154 INFO [RS:1;jenkins-hbase4:34613] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:55:28,155 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:28,155 INFO [RS:1;jenkins-hbase4:34613] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:55:28,155 INFO [RS:1;jenkins-hbase4:34613] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:55:28,156 DEBUG [RS:1;jenkins-hbase4:34613] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x54eff967 to 127.0.0.1:65243 2023-07-19 05:55:28,156 DEBUG [RS:1;jenkins-hbase4:34613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,158 INFO [RS:1;jenkins-hbase4:34613] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34613 2023-07-19 05:55:28,170 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,170 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,172 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,175 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,177 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00018, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,177 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:55:28,177 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:28,177 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,177 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,178 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:28,178 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,178 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:28,178 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00017, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,179 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,179 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:55:28,179 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34613,1689746121857 2023-07-19 05:55:28,179 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,179 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,183 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34613,1689746121857] 2023-07-19 05:55:28,183 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34613,1689746121857; numProcessing=1 2023-07-19 05:55:28,185 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34613,1689746121857 already deleted, retry=false 2023-07-19 05:55:28,185 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34613,1689746121857 expired; onlineServers=3 2023-07-19 05:55:28,204 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/.tmp/cf1/56ea8d42fc7f467a8b4c5beb4118c066 2023-07-19 05:55:28,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/.tmp/cf1/f3d6786e81f546b7b78459c70b2928c5 2023-07-19 05:55:28,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/.tmp/info/12ba01b16a514d88854eada26bfe99ee 2023-07-19 05:55:28,260 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,262 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/info/adcf10598c134d8f9bf188f7c94885aa 2023-07-19 05:55:28,282 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,287 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,288 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:28,288 INFO [RS:1;jenkins-hbase4:34613] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34613,1689746121857; zookeeper connection closed. 2023-07-19 05:55:28,288 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:34613-0x1017c2601c00009, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:28,290 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@77472ab5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@77472ab5 2023-07-19 05:55:28,300 DEBUG [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1504): Waiting on 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:28,303 DEBUG [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1504): Waiting on 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:28,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/.tmp/info/12ba01b16a514d88854eada26bfe99ee as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info/12ba01b16a514d88854eada26bfe99ee 2023-07-19 05:55:28,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/.tmp/cf1/56ea8d42fc7f467a8b4c5beb4118c066 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/56ea8d42fc7f467a8b4c5beb4118c066 2023-07-19 05:55:28,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/.tmp/cf1/f3d6786e81f546b7b78459c70b2928c5 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/f3d6786e81f546b7b78459c70b2928c5 2023-07-19 05:55:28,313 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/56ea8d42fc7f467a8b4c5beb4118c066, entries=100, sequenceid=7, filesize=8.3 K 2023-07-19 05:55:28,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/f3d6786e81f546b7b78459c70b2928c5, entries=200, sequenceid=9, filesize=12.2 K 2023-07-19 05:55:28,326 DEBUG [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1504): Waiting on 1588230740, ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:28,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 2054d9ec275367d29389fbedb9c09867 in 233ms, sequenceid=7, compaction requested=false 2023-07-19 05:55:28,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 168d6c8e660cdb03a471b9cc5e7a3e2b in 218ms, sequenceid=9, compaction requested=false 2023-07-19 05:55:28,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info/12ba01b16a514d88854eada26bfe99ee, entries=2, sequenceid=6, filesize=4.8 K 2023-07-19 05:55:28,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for ecfca84342d05307f433bda44290dd5a in 219ms, sequenceid=6, compaction requested=false 2023-07-19 05:55:28,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-19 05:55:28,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-07-19 05:55:28,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-19 05:55:28,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:28,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:55:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:55:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:28,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:28,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:55:28,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:28,366 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,367 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/table/5ac4cf9a6573436f8499f24426169c83 2023-07-19 05:55:28,373 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/info/adcf10598c134d8f9bf188f7c94885aa as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/adcf10598c134d8f9bf188f7c94885aa 2023-07-19 05:55:28,380 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/adcf10598c134d8f9bf188f7c94885aa, entries=30, sequenceid=19, filesize=8.1 K 2023-07-19 05:55:28,382 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/table/5ac4cf9a6573436f8499f24426169c83 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table/5ac4cf9a6573436f8499f24426169c83 2023-07-19 05:55:28,387 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,391 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table/5ac4cf9a6573436f8499f24426169c83, entries=6, sequenceid=19, filesize=4.8 K 2023-07-19 05:55:28,392 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 266ms, sequenceid=19, compaction requested=false 2023-07-19 05:55:28,399 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,402 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-19 05:55:28,403 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-19 05:55:28,403 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:55:28,403 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:55:28,403 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-19 05:55:28,417 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,450 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-19 05:55:28,450 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-19 05:55:28,460 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-19 05:55:28,460 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-19 05:55:28,469 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-19 05:55:28,469 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-19 05:55:28,471 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,494 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,500 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45061,1689746121819; all regions closed. 2023-07-19 05:55:28,503 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43495,1689746121886; all regions closed. 2023-07-19 05:55:28,508 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,513 DEBUG [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,513 INFO [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45061%2C1689746121819.jenkins-hbase4.apache.org%2C45061%2C1689746121819.regiongroup-1:(num 1689746125415) 2023-07-19 05:55:28,516 DEBUG [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,516 INFO [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43495%2C1689746121886.jenkins-hbase4.apache.org%2C43495%2C1689746121886.regiongroup-1:(num 1689746126031) 2023-07-19 05:55:28,526 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42043,1689746121940; all regions closed. 2023-07-19 05:55:28,539 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,539 DEBUG [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,539 INFO [RS:0;jenkins-hbase4:45061] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45061%2C1689746121819.jenkins-hbase4.apache.org%2C45061%2C1689746121819.regiongroup-0:(num 1689746122573) 2023-07-19 05:55:28,539 DEBUG [RS:0;jenkins-hbase4:45061] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,539 INFO [RS:0;jenkins-hbase4:45061] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,540 INFO [RS:0;jenkins-hbase4:45061] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-19 05:55:28,540 DEBUG [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,540 INFO [RS:2;jenkins-hbase4:43495] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43495%2C1689746121886.jenkins-hbase4.apache.org%2C43495%2C1689746121886.regiongroup-0:(num 1689746122601) 2023-07-19 05:55:28,540 DEBUG [RS:2;jenkins-hbase4:43495] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,540 INFO [RS:2;jenkins-hbase4:43495] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,540 INFO [RS:2;jenkins-hbase4:43495] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-19 05:55:28,543 INFO [RS:0;jenkins-hbase4:45061] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:55:28,544 INFO [RS:2;jenkins-hbase4:43495] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:55:28,544 INFO [RS:0;jenkins-hbase4:45061] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:55:28,544 INFO [RS:0;jenkins-hbase4:45061] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:55:28,544 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:28,544 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:28,544 INFO [RS:2;jenkins-hbase4:43495] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:55:28,544 INFO [RS:2;jenkins-hbase4:43495] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:55:28,544 DEBUG [RS:0;jenkins-hbase4:45061] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d184fa0 to 127.0.0.1:65243 2023-07-19 05:55:28,544 DEBUG [RS:2;jenkins-hbase4:43495] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c59dda5 to 127.0.0.1:65243 2023-07-19 05:55:28,545 DEBUG [RS:0;jenkins-hbase4:45061] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,545 DEBUG [RS:2;jenkins-hbase4:43495] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,546 INFO [RS:0;jenkins-hbase4:45061] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45061 2023-07-19 05:55:28,551 INFO [RS:2;jenkins-hbase4:43495] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43495 2023-07-19 05:55:28,551 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:28,551 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:28,551 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,551 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45061,1689746121819 2023-07-19 05:55:28,552 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00017, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,552 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:55:28,553 DEBUG [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,553 INFO [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42043%2C1689746121940.meta:.meta(num 1689746122824) 2023-07-19 05:55:28,558 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00018, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,558 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:55:28,565 DEBUG [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,565 INFO [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42043%2C1689746121940.jenkins-hbase4.apache.org%2C42043%2C1689746121940.regiongroup-0:(num 1689746122596) 2023-07-19 05:55:28,573 DEBUG [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:55:28,573 INFO [RS:3;jenkins-hbase4:42043] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42043%2C1689746121940.jenkins-hbase4.apache.org%2C42043%2C1689746121940.regiongroup-1:(num 1689746123387) 2023-07-19 05:55:28,573 DEBUG [RS:3;jenkins-hbase4:42043] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,573 INFO [RS:3;jenkins-hbase4:42043] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:28,574 INFO [RS:3;jenkins-hbase4:42043] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-19 05:55:28,574 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:28,574 DEBUG [RS:3;jenkins-hbase4:42043] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ec0fb49 to 127.0.0.1:65243 2023-07-19 05:55:28,574 DEBUG [RS:3;jenkins-hbase4:42043] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,575 INFO [RS:3;jenkins-hbase4:42043] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42043 2023-07-19 05:55:28,580 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,599 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,614 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,644 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,650 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:28,650 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,651 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:28,650 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43495,1689746121886 2023-07-19 05:55:28,651 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,42043,1689746121940 2023-07-19 05:55:28,650 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00018, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:28,651 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45061,1689746121819] 2023-07-19 05:55:28,651 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:55:28,651 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45061,1689746121819; numProcessing=2 2023-07-19 05:55:28,653 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45061,1689746121819 already deleted, retry=false 2023-07-19 05:55:28,654 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45061,1689746121819 expired; onlineServers=2 2023-07-19 05:55:28,654 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43495,1689746121886] 2023-07-19 05:55:28,654 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43495,1689746121886; numProcessing=3 2023-07-19 05:55:28,655 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43495,1689746121886 already deleted, retry=false 2023-07-19 05:55:28,655 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43495,1689746121886 expired; onlineServers=1 2023-07-19 05:55:28,655 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42043,1689746121940] 2023-07-19 05:55:28,655 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42043,1689746121940; numProcessing=4 2023-07-19 05:55:28,656 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,42043,1689746121940 already deleted, retry=false 2023-07-19 05:55:28,657 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42043,1689746121940 expired; onlineServers=0 2023-07-19 05:55:28,657 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36183,1689746121767' ***** 2023-07-19 05:55:28,657 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-19 05:55:28,658 DEBUG [M:0;jenkins-hbase4:36183] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a64e3ca, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:28,658 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36183,1689746121767 2023-07-19 05:55:28,658 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36183,1689746121767; all regions closed. 2023-07-19 05:55:28,658 DEBUG [M:0;jenkins-hbase4:36183] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:28,658 DEBUG [M:0;jenkins-hbase4:36183] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-19 05:55:28,658 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-19 05:55:28,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746122249] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746122249,5,FailOnTimeoutGroup] 2023-07-19 05:55:28,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746122249] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746122249,5,FailOnTimeoutGroup] 2023-07-19 05:55:28,658 DEBUG [M:0;jenkins-hbase4:36183] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-19 05:55:28,661 INFO [M:0;jenkins-hbase4:36183] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-19 05:55:28,661 INFO [M:0;jenkins-hbase4:36183] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-19 05:55:28,661 INFO [M:0;jenkins-hbase4:36183] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-19 05:55:28,661 DEBUG [M:0;jenkins-hbase4:36183] master.HMaster(1512): Stopping service threads 2023-07-19 05:55:28,661 INFO [M:0;jenkins-hbase4:36183] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-19 05:55:28,662 ERROR [M:0;jenkins-hbase4:36183] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-19 05:55:28,662 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-19 05:55:28,663 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:28,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:28,663 INFO [M:0;jenkins-hbase4:36183] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-19 05:55:28,663 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-19 05:55:28,664 DEBUG [M:0;jenkins-hbase4:36183] zookeeper.ZKUtil(398): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-19 05:55:28,664 WARN [M:0;jenkins-hbase4:36183] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-19 05:55:28,664 INFO [M:0;jenkins-hbase4:36183] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-19 05:55:28,665 INFO [M:0;jenkins-hbase4:36183] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-19 05:55:28,665 DEBUG [M:0;jenkins-hbase4:36183] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:55:28,666 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:28,666 DEBUG [M:0;jenkins-hbase4:36183] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:28,666 DEBUG [M:0;jenkins-hbase4:36183] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:55:28,666 DEBUG [M:0;jenkins-hbase4:36183] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:28,666 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.38 KB heapSize=65.18 KB 2023-07-19 05:55:28,687 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,688 INFO [M:0;jenkins-hbase4:36183] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.38 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7d3977543fb84f4c9cc62c6480faff0b 2023-07-19 05:55:28,696 DEBUG [M:0;jenkins-hbase4:36183] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7d3977543fb84f4c9cc62c6480faff0b as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d3977543fb84f4c9cc62c6480faff0b 2023-07-19 05:55:28,703 INFO [M:0;jenkins-hbase4:36183] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d3977543fb84f4c9cc62c6480faff0b, entries=14, sequenceid=116, filesize=8.1 K 2023-07-19 05:55:28,704 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegion(2948): Finished flush of dataSize ~55.38 KB/56708, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 38ms, sequenceid=116, compaction requested=false 2023-07-19 05:55:28,707 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:28,707 DEBUG [M:0;jenkins-hbase4:36183] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:28,707 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,712 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 not finished, retry = 0 2023-07-19 05:55:28,719 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,750 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,793 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,813 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:28,813 INFO [M:0;jenkins-hbase4:36183] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-19 05:55:28,813 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,814 INFO [M:0;jenkins-hbase4:36183] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36183 2023-07-19 05:55:28,817 DEBUG [M:0;jenkins-hbase4:36183] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,36183,1689746121767 already deleted, retry=false 2023-07-19 05:55:28,824 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,862 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,874 INFO [jenkins-hbase4:43215Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 200, current progress: walGroup [jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1]: currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008 at position: 600 walGroup [jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0]: currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 at position: 21752 2023-07-19 05:55:28,875 INFO [jenkins-hbase4:43759Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 100, current progress: walGroup [jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1]: currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109 at position: 10975 walGroup [jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0]: currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 at position: -1 2023-07-19 05:55:28,900 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:28,920 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:28,929 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:28,968 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:28,990 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:28,991 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:36183-0x1017c2601c00007, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:28,991 INFO [M:0;jenkins-hbase4:36183] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36183,1689746121767; zookeeper connection closed. 2023-07-19 05:55:29,008 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,029 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:29,040 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:29,074 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:29,091 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,091 INFO [RS:2;jenkins-hbase4:43495] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43495,1689746121886; zookeeper connection closed. 2023-07-19 05:55:29,091 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:43495-0x1017c2601c0000a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,093 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@27cb3bcf] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@27cb3bcf 2023-07-19 05:55:29,114 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,137 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:29,147 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:29,178 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:29,191 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,191 INFO [RS:3;jenkins-hbase4:42043] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42043,1689746121940; zookeeper connection closed. 2023-07-19 05:55:29,191 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42043-0x1017c2601c0000b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,192 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3ca08b8e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3ca08b8e 2023-07-19 05:55:29,219 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,241 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-19 05:55:29,251 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-19 05:55:29,285 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-19 05:55:29,291 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,291 INFO [RS:0;jenkins-hbase4:45061] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45061,1689746121819; zookeeper connection closed. 2023-07-19 05:55:29,291 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:45061-0x1017c2601c00008, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,293 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@323a66d9] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@323a66d9 2023-07-19 05:55:29,293 INFO [Listener at localhost/45003] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-19 05:55:29,318 INFO [Listener at localhost/45003] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-19 05:55:29,318 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x75fbe421 to 127.0.0.1:65243 2023-07-19 05:55:29,318 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,319 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-19 05:55:29,319 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(257): Found active master hash=1739555004, stopped=false 2023-07-19 05:55:29,319 INFO [Listener at localhost/45003] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:29,321 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-19 05:55:29,321 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-19 05:55:29,322 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-19 05:55:29,322 INFO [Listener at localhost/45003] procedure2.ProcedureExecutor(629): Stopping 2023-07-19 05:55:29,322 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:29,323 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x08e0a71d to 127.0.0.1:65243 2023-07-19 05:55:29,323 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,323 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43759,1689746117439' ***** 2023-07-19 05:55:29,323 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:55:29,323 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43215,1689746117482' ***** 2023-07-19 05:55:29,323 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:55:29,331 INFO [RS:0;jenkins-hbase4:43759] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:55:29,331 INFO [RS:0;jenkins-hbase4:43759] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:55:29,331 INFO [RS:0;jenkins-hbase4:43759] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:55:29,332 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(3305): Received CLOSE for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:29,331 INFO [RS:1;jenkins-hbase4:43215] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:55:29,331 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:55:29,333 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,337 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:29,339 INFO [RS:1;jenkins-hbase4:43215] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:55:29,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8c7505042d76fb8c47639ac419e6a996, disabling compactions & flushes 2023-07-19 05:55:29,340 INFO [RS:1;jenkins-hbase4:43215] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:55:29,339 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:55:29,340 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(3305): Received CLOSE for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:55:29,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:29,339 DEBUG [RS:0;jenkins-hbase4:43759] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4458162a to 127.0.0.1:65243 2023-07-19 05:55:29,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:29,341 DEBUG [RS:0;jenkins-hbase4:43759] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. after waiting 0 ms 2023-07-19 05:55:29,341 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(3305): Received CLOSE for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:29,341 INFO [RS:0;jenkins-hbase4:43759] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:55:29,342 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:29,342 DEBUG [RS:1;jenkins-hbase4:43215] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29f89bf0 to 127.0.0.1:65243 2023-07-19 05:55:29,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 13c92efbf45fed23cebac57cf655e861, disabling compactions & flushes 2023-07-19 05:55:29,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:29,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:29,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:29,343 DEBUG [RS:1;jenkins-hbase4:43215] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8c7505042d76fb8c47639ac419e6a996 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-19 05:55:29,342 INFO [RS:0;jenkins-hbase4:43759] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:55:29,344 INFO [RS:0;jenkins-hbase4:43759] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:55:29,344 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-19 05:55:29,343 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-19 05:55:29,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. after waiting 0 ms 2023-07-19 05:55:29,344 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1478): Online Regions={13c92efbf45fed23cebac57cf655e861=hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861., 8f71d242d3ffe63977903bdc0b5b9fe2=t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.} 2023-07-19 05:55:29,344 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:29,344 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 13c92efbf45fed23cebac57cf655e861 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-19 05:55:29,344 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1504): Waiting on 13c92efbf45fed23cebac57cf655e861, 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:29,347 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:29,350 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-19 05:55:29,351 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1478): Online Regions={8c7505042d76fb8c47639ac419e6a996=t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996., 1588230740=hbase:meta,,1.1588230740} 2023-07-19 05:55:29,351 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1504): Waiting on 1588230740, 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:29,358 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:29,367 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:55:29,368 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:55:29,368 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:55:29,368 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:55:29,368 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:55:29,368 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-19 05:55:29,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-07-19 05:55:29,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-19 05:55:29,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-07-19 05:55:29,371 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=28400, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-19 05:55:29,376 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-07-19 05:55:29,376 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,376 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-07-19 05:55:29,376 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008, lastWalPosition=845, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:29,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/cf1/8755977c20064a56a422f0f1b8af659c 2023-07-19 05:55:29,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:29,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:29,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:29,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/.tmp/info/291551e7f9204b53bba847f884f25d46 2023-07-19 05:55:29,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8755977c20064a56a422f0f1b8af659c 2023-07-19 05:55:29,497 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-19 05:55:29,497 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-19 05:55:29,498 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-19 05:55:29,498 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109, lastWalPosition=14562, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-19 05:55:29,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/.tmp/info/291551e7f9204b53bba847f884f25d46 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info/291551e7f9204b53bba847f884f25d46 2023-07-19 05:55:29,511 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info/291551e7f9204b53bba847f884f25d46, entries=2, sequenceid=6, filesize=4.8 K 2023-07-19 05:55:29,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 13c92efbf45fed23cebac57cf655e861 in 168ms, sequenceid=6, compaction requested=false 2023-07-19 05:55:29,517 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1124 2023-07-19 05:55:29,517 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,517 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1124 2023-07-19 05:55:29,528 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-07-19 05:55:29,538 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/info/ee0fa4692c5f40eca008529c87cbb8d9 2023-07-19 05:55:29,545 DEBUG [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1504): Waiting on 13c92efbf45fed23cebac57cf655e861, 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:55:29,551 DEBUG [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1504): Waiting on 1588230740, 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:55:29,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-19 05:55:29,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:29,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 13c92efbf45fed23cebac57cf655e861: 2023-07-19 05:55:29,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:55:29,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8f71d242d3ffe63977903bdc0b5b9fe2, disabling compactions & flushes 2023-07-19 05:55:29,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:29,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:29,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. after waiting 0 ms 2023-07-19 05:55:29,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:29,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8f71d242d3ffe63977903bdc0b5b9fe2 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-19 05:55:29,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/norep/1214f5f573c94782b0b51c9f3fc07563 2023-07-19 05:55:29,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1214f5f573c94782b0b51c9f3fc07563 2023-07-19 05:55:29,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/cf1/8755977c20064a56a422f0f1b8af659c as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1/8755977c20064a56a422f0f1b8af659c 2023-07-19 05:55:29,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8755977c20064a56a422f0f1b8af659c 2023-07-19 05:55:29,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1/8755977c20064a56a422f0f1b8af659c, entries=100, sequenceid=106, filesize=8.2 K 2023-07-19 05:55:29,593 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/rep_barrier/44f4e872a3654b3e9e31e378b714b20c 2023-07-19 05:55:29,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/norep/1214f5f573c94782b0b51c9f3fc07563 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep/1214f5f573c94782b0b51c9f3fc07563 2023-07-19 05:55:29,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1214f5f573c94782b0b51c9f3fc07563 2023-07-19 05:55:29,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep/1214f5f573c94782b0b51c9f3fc07563, entries=51, sequenceid=106, filesize=6.5 K 2023-07-19 05:55:29,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for 8c7505042d76fb8c47639ac419e6a996 in 260ms, sequenceid=106, compaction requested=false 2023-07-19 05:55:29,622 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14871 2023-07-19 05:55:29,622 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,622 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14871 2023-07-19 05:55:29,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/cf1/5ed5951d0933423da886fa0a7efe3914 2023-07-19 05:55:29,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ed5951d0933423da886fa0a7efe3914 2023-07-19 05:55:29,630 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,631 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-19 05:55:29,631 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-19 05:55:29,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:29,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8c7505042d76fb8c47639ac419e6a996: 2023-07-19 05:55:29,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:55:29,645 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-07-19 05:55:29,645 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,645 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-07-19 05:55:29,650 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/table/ddf6019c76084b638ea6c99ae82d5c5f 2023-07-19 05:55:29,658 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/info/ee0fa4692c5f40eca008529c87cbb8d9 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info/ee0fa4692c5f40eca008529c87cbb8d9 2023-07-19 05:55:29,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/norep/938d1f6c4e354a2a94f0ac8447e5a2c3 2023-07-19 05:55:29,665 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info/ee0fa4692c5f40eca008529c87cbb8d9, entries=30, sequenceid=19, filesize=8.1 K 2023-07-19 05:55:29,667 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/rep_barrier/44f4e872a3654b3e9e31e378b714b20c as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier/44f4e872a3654b3e9e31e378b714b20c 2023-07-19 05:55:29,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 938d1f6c4e354a2a94f0ac8447e5a2c3 2023-07-19 05:55:29,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/cf1/5ed5951d0933423da886fa0a7efe3914 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1/5ed5951d0933423da886fa0a7efe3914 2023-07-19 05:55:29,677 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier/44f4e872a3654b3e9e31e378b714b20c, entries=2, sequenceid=19, filesize=4.9 K 2023-07-19 05:55:29,678 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/table/ddf6019c76084b638ea6c99ae82d5c5f as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table/ddf6019c76084b638ea6c99ae82d5c5f 2023-07-19 05:55:29,685 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table/ddf6019c76084b638ea6c99ae82d5c5f, entries=6, sequenceid=19, filesize=4.8 K 2023-07-19 05:55:29,686 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 318ms, sequenceid=19, compaction requested=false 2023-07-19 05:55:29,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ed5951d0933423da886fa0a7efe3914 2023-07-19 05:55:29,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1/5ed5951d0933423da886fa0a7efe3914, entries=200, sequenceid=206, filesize=11.9 K 2023-07-19 05:55:29,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/norep/938d1f6c4e354a2a94f0ac8447e5a2c3 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep/938d1f6c4e354a2a94f0ac8447e5a2c3 2023-07-19 05:55:29,706 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-19 05:55:29,707 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-19 05:55:29,708 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:55:29,708 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:55:29,708 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-19 05:55:29,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 938d1f6c4e354a2a94f0ac8447e5a2c3 2023-07-19 05:55:29,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep/938d1f6c4e354a2a94f0ac8447e5a2c3, entries=101, sequenceid=206, filesize=8.3 K 2023-07-19 05:55:29,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 8f71d242d3ffe63977903bdc0b5b9fe2 in 157ms, sequenceid=206, compaction requested=false 2023-07-19 05:55:29,727 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-19 05:55:29,728 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-19 05:55:29,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8f71d242d3ffe63977903bdc0b5b9fe2: 2023-07-19 05:55:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:55:29,736 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:29,745 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43215,1689746117482; all regions closed. 2023-07-19 05:55:29,751 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43759,1689746117439; all regions closed. 2023-07-19 05:55:29,753 DEBUG [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:55:29,753 INFO [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1:(num 1689746120008) 2023-07-19 05:55:29,762 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 not finished, retry = 0 2023-07-19 05:55:29,764 DEBUG [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:55:29,764 INFO [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43759%2C1689746117439.meta:.meta(num 1689746119158) 2023-07-19 05:55:29,772 DEBUG [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:55:29,772 INFO [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0:(num 1689746118997) 2023-07-19 05:55:29,779 DEBUG [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:55:29,779 INFO [RS:0;jenkins-hbase4:43759] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1:(num 1689746124109) 2023-07-19 05:55:29,779 DEBUG [RS:0;jenkins-hbase4:43759] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,779 INFO [RS:0;jenkins-hbase4:43759] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:29,779 INFO [RS:0;jenkins-hbase4:43759] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-19 05:55:29,779 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:29,779 INFO [RS:0;jenkins-hbase4:43759] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-19 05:55:29,838 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 was moved to hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 2023-07-19 05:55:29,863 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-19 05:55:29,865 DEBUG [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:55:29,865 INFO [RS:1;jenkins-hbase4:43215] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0:(num 1689746118997) 2023-07-19 05:55:29,865 DEBUG [RS:1;jenkins-hbase4:43215] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,865 INFO [RS:1;jenkins-hbase4:43215] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:55:29,865 INFO [RS:1;jenkins-hbase4:43215] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-19 05:55:29,865 INFO [RS:1;jenkins-hbase4:43215] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:55:29,865 INFO [RS:1;jenkins-hbase4:43215] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:55:29,865 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:29,865 INFO [RS:1;jenkins-hbase4:43215] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:55:29,866 INFO [RS:1;jenkins-hbase4:43215] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-19 05:55:29,868 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, it was closed cleanly currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 at position: 29258 2023-07-19 05:55:29,868 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 2023-07-19 05:55:29,868 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:29,882 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00018, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,882 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00018, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,882 DEBUG [RS:0;jenkins-hbase4:43759] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x562919b5 to 127.0.0.1:65243 2023-07-19 05:55:29,882 DEBUG [RS:0;jenkins-hbase4:43759] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:29,982 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00017, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,982 DEBUG [RS:1;jenkins-hbase4:43215] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a77650e to 127.0.0.1:65243 2023-07-19 05:55:29,982 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00017, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:29,982 DEBUG [RS:1;jenkins-hbase4:43215] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:30,083 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:381) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) 2023-07-19 05:55:30,083 INFO [RS:0;jenkins-hbase4:43759] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1 terminated 2023-07-19 05:55:30,084 INFO [RS:0;jenkins-hbase4:43759] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1 terminated 2023-07-19 05:55:30,085 INFO [RS:0;jenkins-hbase4:43759] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43759 2023-07-19 05:55:30,087 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:30,087 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:30,087 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:55:30,087 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:55:30,087 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:55:30,089 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43759,1689746117439] 2023-07-19 05:55:30,089 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43759,1689746117439; numProcessing=1 2023-07-19 05:55:30,093 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,43759,1689746117439 already deleted, retry=false 2023-07-19 05:55:30,093 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43759,1689746117439 expired; onlineServers=1 2023-07-19 05:55:30,184 INFO [RS:1;jenkins-hbase4:43215] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1 terminated 2023-07-19 05:55:30,185 INFO [RS:1;jenkins-hbase4:43215] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1 terminated 2023-07-19 05:55:30,186 INFO [RS:1;jenkins-hbase4:43215] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43215 2023-07-19 05:55:30,188 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:30,188 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:55:30,189 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:30,189 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43215,1689746117482; numProcessing=2 2023-07-19 05:55:30,191 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,43215,1689746117482 already deleted, retry=false 2023-07-19 05:55:30,191 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43215,1689746117482 expired; onlineServers=0 2023-07-19 05:55:30,191 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39519,1689746116265' ***** 2023-07-19 05:55:30,191 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-19 05:55:30,191 DEBUG [M:0;jenkins-hbase4:39519] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@26928a87, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:30,192 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39519,1689746116265 2023-07-19 05:55:30,192 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39519,1689746116265; all regions closed. 2023-07-19 05:55:30,192 DEBUG [M:0;jenkins-hbase4:39519] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:30,192 DEBUG [M:0;jenkins-hbase4:39519] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-19 05:55:30,192 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-19 05:55:30,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746118639] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746118639,5,FailOnTimeoutGroup] 2023-07-19 05:55:30,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746118636] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746118636,5,FailOnTimeoutGroup] 2023-07-19 05:55:30,192 DEBUG [M:0;jenkins-hbase4:39519] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-19 05:55:30,193 INFO [M:0;jenkins-hbase4:39519] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-19 05:55:30,193 INFO [M:0;jenkins-hbase4:39519] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-19 05:55:30,194 INFO [M:0;jenkins-hbase4:39519] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-19 05:55:30,194 DEBUG [M:0;jenkins-hbase4:39519] master.HMaster(1512): Stopping service threads 2023-07-19 05:55:30,194 INFO [M:0;jenkins-hbase4:39519] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-19 05:55:30,194 ERROR [M:0;jenkins-hbase4:39519] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-19 05:55:30,194 INFO [M:0;jenkins-hbase4:39519] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-19 05:55:30,195 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-19 05:55:30,196 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-19 05:55:30,196 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:30,196 DEBUG [M:0;jenkins-hbase4:39519] zookeeper.ZKUtil(398): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-19 05:55:30,196 WARN [M:0;jenkins-hbase4:39519] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-19 05:55:30,196 INFO [M:0;jenkins-hbase4:39519] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-19 05:55:30,196 INFO [M:0;jenkins-hbase4:39519] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-19 05:55:30,197 DEBUG [M:0;jenkins-hbase4:39519] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:55:30,197 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:30,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:55:30,197 DEBUG [M:0;jenkins-hbase4:39519] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:30,197 DEBUG [M:0;jenkins-hbase4:39519] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:55:30,197 DEBUG [M:0;jenkins-hbase4:39519] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:30,198 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.02 KB heapSize=72.60 KB 2023-07-19 05:55:30,215 INFO [M:0;jenkins-hbase4:39519] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.02 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/25c9c9ede8c1485a853e23ea9291ba92 2023-07-19 05:55:30,221 DEBUG [M:0;jenkins-hbase4:39519] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/25c9c9ede8c1485a853e23ea9291ba92 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/25c9c9ede8c1485a853e23ea9291ba92 2023-07-19 05:55:30,226 INFO [M:0;jenkins-hbase4:39519] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/25c9c9ede8c1485a853e23ea9291ba92, entries=17, sequenceid=136, filesize=8.5 K 2023-07-19 05:55:30,227 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegion(2948): Finished flush of dataSize ~61.02 KB/62484, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=136, compaction requested=false 2023-07-19 05:55:30,229 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:55:30,229 DEBUG [M:0;jenkins-hbase4:39519] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:30,233 INFO [M:0;jenkins-hbase4:39519] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-19 05:55:30,233 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:55:30,234 INFO [M:0;jenkins-hbase4:39519] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39519 2023-07-19 05:55:30,235 DEBUG [M:0;jenkins-hbase4:39519] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,39519,1689746116265 already deleted, retry=false 2023-07-19 05:55:30,262 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:55:30,283 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:30,283 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43759-0x1017c2601c00001, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:30,283 INFO [RS:0;jenkins-hbase4:43759] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43759,1689746117439; zookeeper connection closed. 2023-07-19 05:55:30,283 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@275f4f94] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@275f4f94 2023-07-19 05:55:30,383 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:30,383 INFO [M:0;jenkins-hbase4:39519] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39519,1689746116265; zookeeper connection closed. 2023-07-19 05:55:30,383 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): master:39519-0x1017c2601c00000, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:30,483 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:30,483 INFO [RS:1;jenkins-hbase4:43215] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43215,1689746117482; zookeeper connection closed. 2023-07-19 05:55:30,483 DEBUG [Listener at localhost/33433-EventThread] zookeeper.ZKWatcher(600): regionserver:43215-0x1017c2601c00002, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:30,484 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@66b46a6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@66b46a6 2023-07-19 05:55:30,484 INFO [Listener at localhost/45003] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:30,486 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:30,487 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34645 2023-07-19 05:55:30,487 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:30,489 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:30,490 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34645 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:30,493 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:346450x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:30,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34645-0x1017c2601c0001d connected 2023-07-19 05:55:30,496 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:30,497 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:30,497 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:30,499 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34645 2023-07-19 05:55:30,499 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34645 2023-07-19 05:55:30,499 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34645 2023-07-19 05:55:30,500 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34645 2023-07-19 05:55:30,500 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34645 2023-07-19 05:55:30,500 INFO [Listener at localhost/45003] master.HMaster(444): hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881, hbase.cluster.distributed=false 2023-07-19 05:55:30,501 DEBUG [pool-462-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-19 05:55:30,512 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:55:30,513 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:30,513 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:30,513 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:30,513 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:30,513 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:30,513 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:30,514 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38293 2023-07-19 05:55:30,514 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:30,516 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:30,516 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:30,517 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:30,518 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38293 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:30,522 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:382930x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:30,523 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:382930x0, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:55:30,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38293-0x1017c2601c0001e connected 2023-07-19 05:55:30,524 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:55:30,524 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:55:30,524 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38293 2023-07-19 05:55:30,525 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38293 2023-07-19 05:55:30,525 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38293 2023-07-19 05:55:30,525 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38293 2023-07-19 05:55:30,525 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38293 2023-07-19 05:55:30,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,528 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-19 05:55:30,529 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,530 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:30,530 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:55:30,530 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:30,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-19 05:55:30,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-19 05:55:30,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,34645,1689746130485 from backup master directory 2023-07-19 05:55:30,534 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,534 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-19 05:55:30,534 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:30,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:30,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x60e35b4e to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:30,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f7814ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:30,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:30,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-19 05:55:30,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:30,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:30,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767 to hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767-dead as it is dead 2023-07-19 05:55:30,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767-dead/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 2023-07-19 05:55:30,591 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767-dead/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 after 4ms 2023-07-19 05:55:30,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767-dead/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 to hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 2023-07-19 05:55:30,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,36183,1689746121767-dead 2023-07-19 05:55:30,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:30,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/oldWALs, maxLogs=10 2023-07-19 05:55:30,610 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:30,610 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:30,610 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:30,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 2023-07-19 05:55:30,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK]] 2023-07-19 05:55:30,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:30,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:30,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:30,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:30,616 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:30,617 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-19 05:55:30,618 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-19 05:55:30,624 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d3977543fb84f4c9cc62c6480faff0b 2023-07-19 05:55:30,624 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:30,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-19 05:55:30,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 2023-07-19 05:55:30,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 2023-07-19 05:55:30,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C36183%2C1689746121767.jenkins-hbase4.apache.org%2C36183%2C1689746121767.regiongroup-0.1689746122133 2023-07-19 05:55:30,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:30,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-19 05:55:30,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67429733, jitterRate=0.004781320691108704}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:30,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:30,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-19 05:55:30,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-19 05:55:30,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-19 05:55:30,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-19 05:55:30,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-19 05:55:30,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:30,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-19 05:55:30,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-19 05:55:30,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-19 05:55:30,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-19 05:55:30,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-19 05:55:30,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 17 msec 2023-07-19 05:55:30,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-19 05:55:30,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-19 05:55:30,661 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42043,1689746121940, table=hbase:meta, region=1588230740 2023-07-19 05:55:30,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-19 05:55:30,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,42043,1689746121940 already deleted, retry=false 2023-07-19 05:55:30,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42043,1689746121940 on jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42043,1689746121940, splitWal=true, meta=true 2023-07-19 05:55:30,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,42043,1689746121940 (carryingMeta=true) jenkins-hbase4.apache.org,42043,1689746121940/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@159313e9[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:55:30,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34613,1689746121857 already deleted, retry=false 2023-07-19 05:55:30,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34613,1689746121857 on jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34613,1689746121857, splitWal=true, meta=false 2023-07-19 05:55:30,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,34613,1689746121857 (carryingMeta=false) jenkins-hbase4.apache.org,34613,1689746121857/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@785c6714[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:55:30,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45061,1689746121819 already deleted, retry=false 2023-07-19 05:55:30,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45061,1689746121819 on jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45061,1689746121819, splitWal=true, meta=false 2023-07-19 05:55:30,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,45061,1689746121819 (carryingMeta=false) jenkins-hbase4.apache.org,45061,1689746121819/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@755d3fad[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:55:30,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43495,1689746121886 already deleted, retry=false 2023-07-19 05:55:30,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43495,1689746121886 on jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43495,1689746121886, splitWal=true, meta=false 2023-07-19 05:55:30,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,43495,1689746121886 (carryingMeta=false) jenkins-hbase4.apache.org,43495,1689746121886/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@156fa827[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:55:30,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-19 05:55:30,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-19 05:55:30,693 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-19 05:55:30,693 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-19 05:55:30,693 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-19 05:55:30,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-19 05:55:30,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-19 05:55:30,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-19 05:55:30,696 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:30,696 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:55:30,696 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:55:30,696 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34645,1689746130485, sessionid=0x1017c2601c0001d, setting cluster-up flag (Was=false) 2023-07-19 05:55:30,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-19 05:55:30,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-19 05:55:30,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:55:30,706 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.hbase-snapshot/.tmp 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:30,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689746160711 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-19 05:55:30,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-19 05:55:30,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-19 05:55:30,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-19 05:55:30,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-19 05:55:30,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-19 05:55:30,716 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746130716,5,FailOnTimeoutGroup] 2023-07-19 05:55:30,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746130717,5,FailOnTimeoutGroup] 2023-07-19 05:55:30,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-19 05:55:30,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,717 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42043,1689746121940; numProcessing=1 2023-07-19 05:55:30,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,718 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43495,1689746121886; numProcessing=2 2023-07-19 05:55:30,718 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45061,1689746121819; numProcessing=3 2023-07-19 05:55:30,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689746130718, completionTime=-1 2023-07-19 05:55:30,718 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34613,1689746121857; numProcessing=4 2023-07-19 05:55:30,718 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-19 05:55:30,718 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34613,1689746121857, splitWal=true, meta=false 2023-07-19 05:55:30,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-19 05:55:30,718 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45061,1689746121819, splitWal=true, meta=false 2023-07-19 05:55:30,718 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42043,1689746121940, splitWal=true, meta=true 2023-07-19 05:55:30,718 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43495,1689746121886, splitWal=true, meta=false 2023-07-19 05:55:30,721 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42043,1689746121940, splitWal=true, meta=true, isMeta: true 2023-07-19 05:55:30,723 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940-splitting 2023-07-19 05:55:30,724 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940-splitting dir is empty, no logs to split. 2023-07-19 05:55:30,724 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42043,1689746121940 WAL count=0, meta=true 2023-07-19 05:55:30,727 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(951): ClusterId : 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:30,727 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:30,727 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940-splitting dir is empty, no logs to split. 2023-07-19 05:55:30,727 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42043,1689746121940 WAL count=0, meta=true 2023-07-19 05:55:30,727 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42043,1689746121940 WAL splitting is done? wals=0, meta=true 2023-07-19 05:55:30,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-19 05:55:30,730 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-19 05:55:30,732 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:55:30,732 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:30,732 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:30,734 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:30,735 DEBUG [RS:0;jenkins-hbase4:38293] zookeeper.ReadOnlyZKClient(139): Connect 0x113728db to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:30,739 DEBUG [RS:0;jenkins-hbase4:38293] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ff0d48d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:30,740 DEBUG [RS:0;jenkins-hbase4:38293] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bcf43ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:30,749 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38293 2023-07-19 05:55:30,749 INFO [RS:0;jenkins-hbase4:38293] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:30,749 INFO [RS:0;jenkins-hbase4:38293] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:30,749 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:30,749 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34645,1689746130485 with isa=jenkins-hbase4.apache.org/172.31.14.131:38293, startcode=1689746130512 2023-07-19 05:55:30,750 DEBUG [RS:0;jenkins-hbase4:38293] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:30,751 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46805, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:30,753 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34645] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:30,753 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:55:30,753 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33839 2023-07-19 05:55:30,753 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:30,755 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:55:30,755 DEBUG [RS:0;jenkins-hbase4:38293] zookeeper.ZKUtil(162): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:30,756 WARN [RS:0;jenkins-hbase4:38293] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:30,756 INFO [RS:0;jenkins-hbase4:38293] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:30,756 INFO [RS:0;jenkins-hbase4:38293] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:30,756 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:30,756 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38293,1689746130512] 2023-07-19 05:55:30,760 DEBUG [RS:0;jenkins-hbase4:38293] zookeeper.ZKUtil(162): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:30,761 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:30,761 INFO [RS:0;jenkins-hbase4:38293] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:30,763 INFO [RS:0;jenkins-hbase4:38293] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:30,763 INFO [RS:0;jenkins-hbase4:38293] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:30,763 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,763 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:30,764 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,764 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,765 DEBUG [RS:0;jenkins-hbase4:38293] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:30,766 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,766 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,766 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,768 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-19 05:55:30,777 INFO [RS:0;jenkins-hbase4:38293] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:30,777 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38293,1689746130512-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:30,788 INFO [RS:0;jenkins-hbase4:38293] regionserver.Replication(203): jenkins-hbase4.apache.org,38293,1689746130512 started 2023-07-19 05:55:30,788 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38293,1689746130512, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38293, sessionid=0x1017c2601c0001e 2023-07-19 05:55:30,788 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:30,788 DEBUG [RS:0;jenkins-hbase4:38293] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:30,788 DEBUG [RS:0;jenkins-hbase4:38293] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38293,1689746130512' 2023-07-19 05:55:30,788 DEBUG [RS:0;jenkins-hbase4:38293] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-19 05:55:30,788 DEBUG [RS:0;jenkins-hbase4:38293] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38293,1689746130512' 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-19 05:55:30,789 DEBUG [RS:0;jenkins-hbase4:38293] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:30,789 INFO [RS:0;jenkins-hbase4:38293] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:30,789 INFO [RS:0;jenkins-hbase4:38293] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:30,882 DEBUG [jenkins-hbase4:34645] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-19 05:55:30,883 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38293,1689746130512, state=OPENING 2023-07-19 05:55:30,886 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-19 05:55:30,886 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-19 05:55:30,886 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38293,1689746130512}] 2023-07-19 05:55:30,889 INFO [RS:0;jenkins-hbase4:38293] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:30,891 INFO [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38293%2C1689746130512.jenkins-hbase4.apache.org%2C38293%2C1689746130512.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:30,906 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:30,906 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:30,906 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:30,910 INFO [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512/jenkins-hbase4.apache.org%2C38293%2C1689746130512.jenkins-hbase4.apache.org%2C38293%2C1689746130512.regiongroup-0.1689746130891 2023-07-19 05:55:30,911 DEBUG [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK]] 2023-07-19 05:55:31,040 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:31,040 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:31,042 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41590, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:31,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-19 05:55:31,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:31,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:31,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:31,047 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38293%2C1689746130512.meta, suffix=.meta, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:31,064 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:31,064 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:55:31,065 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:31,069 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512/jenkins-hbase4.apache.org%2C38293%2C1689746130512.meta.1689746131047.meta 2023-07-19 05:55:31,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK]] 2023-07-19 05:55:31,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:31,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-19 05:55:31,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-19 05:55:31,071 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-19 05:55:31,071 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-19 05:55:31,071 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:31,071 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-19 05:55:31,071 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-19 05:55:31,073 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:55:31,074 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:55:31,074 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:55:31,075 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:55:31,086 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/adcf10598c134d8f9bf188f7c94885aa 2023-07-19 05:55:31,086 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:31,086 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:55:31,087 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:31,087 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:31,088 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:55:31,088 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:31,088 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:55:31,089 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:55:31,089 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:55:31,090 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:55:31,096 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table/5ac4cf9a6573436f8499f24426169c83 2023-07-19 05:55:31,096 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:31,097 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:55:31,097 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:55:31,100 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:55:31,101 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:55:31,102 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64059963, jitterRate=-0.04543216526508331}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:55:31,102 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:55:31,104 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689746131040 2023-07-19 05:55:31,108 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-19 05:55:31,109 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-19 05:55:31,110 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38293,1689746130512, state=OPEN 2023-07-19 05:55:31,111 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-19 05:55:31,112 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-19 05:55:31,115 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-19 05:55:31,115 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38293,1689746130512 in 226 msec 2023-07-19 05:55:31,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=15 2023-07-19 05:55:31,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 387 msec 2023-07-19 05:55:32,271 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-19 05:55:33,773 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-19 05:55:35,212 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:55:35,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-19 05:55:35,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-19 05:55:35,230 DEBUG [hconnection-0xd4985-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:35,232 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41604, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:35,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ecfca84342d05307f433bda44290dd5a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42043,1689746121940, regionLocation=jenkins-hbase4.apache.org,42043,1689746121940, openSeqNum=2 2023-07-19 05:55:35,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=2054d9ec275367d29389fbedb9c09867, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45061,1689746121819, regionLocation=jenkins-hbase4.apache.org,45061,1689746121819, openSeqNum=2 2023-07-19 05:55:35,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43495,1689746121886, regionLocation=jenkins-hbase4.apache.org,43495,1689746121886, openSeqNum=2 2023-07-19 05:55:35,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-19 05:55:35,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689746195243 2023-07-19 05:55:35,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689746255244 2023-07-19 05:55:35,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 17 msec 2023-07-19 05:55:35,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34645,1689746130485-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:35,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34645,1689746130485-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:35,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34645,1689746130485-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:35,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34645, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:35,275 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45061,1689746121819 had 1 regions 2023-07-19 05:55:35,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:35,276 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42043,1689746121940 had 2 regions 2023-07-19 05:55:35,276 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. is NOT online; state={ecfca84342d05307f433bda44290dd5a state=OPEN, ts=1689746135243, server=jenkins-hbase4.apache.org,42043,1689746121940}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-19 05:55:35,275 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43495,1689746121886 had 1 regions 2023-07-19 05:55:35,275 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34613,1689746121857 had 0 regions 2023-07-19 05:55:35,277 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45061,1689746121819, splitWal=true, meta=false, isMeta: false 2023-07-19 05:55:35,278 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42043,1689746121940, splitWal=true, meta=true, isMeta: false 2023-07-19 05:55:35,278 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43495,1689746121886, splitWal=true, meta=false, isMeta: false 2023-07-19 05:55:35,278 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34613,1689746121857, splitWal=true, meta=false, isMeta: false 2023-07-19 05:55:35,279 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819-splitting 2023-07-19 05:55:35,281 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,281 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45061,1689746121819 WAL count=0, meta=false 2023-07-19 05:55:35,282 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886-splitting 2023-07-19 05:55:35,282 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,282 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,42043,1689746121940 WAL count=0, meta=false 2023-07-19 05:55:35,283 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,42043,1689746121940/hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., unknown_server=jenkins-hbase4.apache.org,45061,1689746121819/t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867., unknown_server=jenkins-hbase4.apache.org,43495,1689746121886/t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:35,284 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857-splitting 2023-07-19 05:55:35,284 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,284 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,43495,1689746121886 WAL count=0, meta=false 2023-07-19 05:55:35,285 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,285 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34613,1689746121857 WAL count=0, meta=false 2023-07-19 05:55:35,286 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,286 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45061,1689746121819 WAL count=0, meta=false 2023-07-19 05:55:35,286 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45061,1689746121819 WAL splitting is done? wals=0, meta=false 2023-07-19 05:55:35,287 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,42043,1689746121940-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,287 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,42043,1689746121940 WAL count=0, meta=false 2023-07-19 05:55:35,287 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42043,1689746121940 WAL splitting is done? wals=0, meta=false 2023-07-19 05:55:35,289 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN}] 2023-07-19 05:55:35,289 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,290 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34613,1689746121857 WAL count=0, meta=false 2023-07-19 05:55:35,290 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34613,1689746121857 WAL splitting is done? wals=0, meta=false 2023-07-19 05:55:35,290 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN 2023-07-19 05:55:35,291 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45061,1689746121819 failed, ignore...File hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,45061,1689746121819-splitting does not exist. 2023-07-19 05:55:35,291 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886-splitting dir is empty, no logs to split. 2023-07-19 05:55:35,291 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,43495,1689746121886 WAL count=0, meta=false 2023-07-19 05:55:35,291 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43495,1689746121886 WAL splitting is done? wals=0, meta=false 2023-07-19 05:55:35,291 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:55:35,292 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN}] 2023-07-19 05:55:35,293 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN 2023-07-19 05:55:35,293 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34613,1689746121857 failed, ignore...File hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,34613,1689746121857-splitting does not exist. 2023-07-19 05:55:35,294 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:55:35,294 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43495,1689746121886 failed, ignore...File hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,43495,1689746121886-splitting does not exist. 2023-07-19 05:55:35,295 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34613,1689746121857 after splitting done 2023-07-19 05:55:35,295 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34613,1689746121857 from processing; numProcessing=3 2023-07-19 05:55:35,295 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN}] 2023-07-19 05:55:35,296 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN 2023-07-19 05:55:35,297 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:55:35,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34613,1689746121857, splitWal=true, meta=false in 4.6190 sec 2023-07-19 05:55:35,442 DEBUG [jenkins-hbase4:34645] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-19 05:55:35,445 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=ecfca84342d05307f433bda44290dd5a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:35,445 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:35,445 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=2054d9ec275367d29389fbedb9c09867, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:35,445 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746135444"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746135444"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746135444"}]},"ts":"1689746135444"} 2023-07-19 05:55:35,445 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746135445"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746135445"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746135445"}]},"ts":"1689746135445"} 2023-07-19 05:55:35,445 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746135445"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746135445"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746135445"}]},"ts":"1689746135445"} 2023-07-19 05:55:35,448 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure ecfca84342d05307f433bda44290dd5a, server=jenkins-hbase4.apache.org,38293,1689746130512}] 2023-07-19 05:55:35,450 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=23, state=RUNNABLE; OpenRegionProcedure 168d6c8e660cdb03a471b9cc5e7a3e2b, server=jenkins-hbase4.apache.org,38293,1689746130512}] 2023-07-19 05:55:35,451 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=22, state=RUNNABLE; OpenRegionProcedure 2054d9ec275367d29389fbedb9c09867, server=jenkins-hbase4.apache.org,38293,1689746130512}] 2023-07-19 05:55:35,606 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:35,606 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:35,608 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38293%2C1689746130512.jenkins-hbase4.apache.org%2C38293%2C1689746130512.regiongroup-1, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:55:35,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:35,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 168d6c8e660cdb03a471b9cc5e7a3e2b, NAME => 't2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:35,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:35,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,612 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,614 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1 2023-07-19 05:55:35,614 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1 2023-07-19 05:55:35,614 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 168d6c8e660cdb03a471b9cc5e7a3e2b columnFamilyName cf1 2023-07-19 05:55:35,622 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/f3d6786e81f546b7b78459c70b2928c5 2023-07-19 05:55:35,622 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(310): Store=168d6c8e660cdb03a471b9cc5e7a3e2b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:35,622 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,623 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/norep 2023-07-19 05:55:35,624 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/norep 2023-07-19 05:55:35,624 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:55:35,624 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:55:35,624 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:55:35,624 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 168d6c8e660cdb03a471b9cc5e7a3e2b columnFamilyName norep 2023-07-19 05:55:35,625 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(310): Store=168d6c8e660cdb03a471b9cc5e7a3e2b/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:35,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,637 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512/jenkins-hbase4.apache.org%2C38293%2C1689746130512.jenkins-hbase4.apache.org%2C38293%2C1689746130512.regiongroup-1.1689746135608 2023-07-19 05:55:35,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:55:35,638 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK]] 2023-07-19 05:55:35,639 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ecfca84342d05307f433bda44290dd5a, NAME => 'hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:35,639 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,639 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:35,639 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,639 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:55:35,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 168d6c8e660cdb03a471b9cc5e7a3e2b; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63011567, jitterRate=-0.06105448305606842}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:55:35,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:55:35,643 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b., pid=25, masterSystemTime=1689746135602 2023-07-19 05:55:35,644 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info 2023-07-19 05:55:35,644 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info 2023-07-19 05:55:35,645 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ecfca84342d05307f433bda44290dd5a columnFamilyName info 2023-07-19 05:55:35,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:35,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:55:35,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:35,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2054d9ec275367d29389fbedb9c09867, NAME => 't1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:35,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,646 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:35,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:35,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,647 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746135646"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746135646"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746135646"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746135646"}]},"ts":"1689746135646"} 2023-07-19 05:55:35,648 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,649 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1 2023-07-19 05:55:35,649 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1 2023-07-19 05:55:35,650 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2054d9ec275367d29389fbedb9c09867 columnFamilyName cf1 2023-07-19 05:55:35,652 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info/12ba01b16a514d88854eada26bfe99ee 2023-07-19 05:55:35,652 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(310): Store=ecfca84342d05307f433bda44290dd5a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:35,652 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=23 2023-07-19 05:55:35,652 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=23, state=SUCCESS; OpenRegionProcedure 168d6c8e660cdb03a471b9cc5e7a3e2b, server=jenkins-hbase4.apache.org,38293,1689746130512 in 198 msec 2023-07-19 05:55:35,653 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,655 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=18 2023-07-19 05:55:35,655 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,655 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN in 357 msec 2023-07-19 05:55:35,655 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43495,1689746121886 after splitting done 2023-07-19 05:55:35,655 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43495,1689746121886 from processing; numProcessing=2 2023-07-19 05:55:35,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43495,1689746121886, splitWal=true, meta=false in 4.9730 sec 2023-07-19 05:55:35,658 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:55:35,659 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ecfca84342d05307f433bda44290dd5a; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74926574, jitterRate=0.11649295687675476}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:35,659 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:55:35,663 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., pid=24, masterSystemTime=1689746135602 2023-07-19 05:55:35,665 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:35,665 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:55:35,665 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=ecfca84342d05307f433bda44290dd5a, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:35,665 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746135665"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746135665"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746135665"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746135665"}]},"ts":"1689746135665"} 2023-07-19 05:55:35,666 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/56ea8d42fc7f467a8b4c5beb4118c066 2023-07-19 05:55:35,666 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(310): Store=2054d9ec275367d29389fbedb9c09867/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:35,666 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,668 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/norep 2023-07-19 05:55:35,668 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/norep 2023-07-19 05:55:35,668 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2054d9ec275367d29389fbedb9c09867 columnFamilyName norep 2023-07-19 05:55:35,669 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-19 05:55:35,669 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(310): Store=2054d9ec275367d29389fbedb9c09867/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:35,669 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure ecfca84342d05307f433bda44290dd5a, server=jenkins-hbase4.apache.org,38293,1689746130512 in 219 msec 2023-07-19 05:55:35,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,671 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=15 2023-07-19 05:55:35,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,671 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN in 380 msec 2023-07-19 05:55:35,671 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42043,1689746121940 after splitting done 2023-07-19 05:55:35,671 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42043,1689746121940 from processing; numProcessing=1 2023-07-19 05:55:35,673 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42043,1689746121940, splitWal=true, meta=true in 5.0020 sec 2023-07-19 05:55:35,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:55:35,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:55:35,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2054d9ec275367d29389fbedb9c09867; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59171232, jitterRate=-0.11827993392944336}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:55:35,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:55:35,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867., pid=26, masterSystemTime=1689746135602 2023-07-19 05:55:35,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:35,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:55:35,679 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=2054d9ec275367d29389fbedb9c09867, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:55:35,679 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746135679"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746135679"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746135679"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746135679"}]},"ts":"1689746135679"} 2023-07-19 05:55:35,682 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=22 2023-07-19 05:55:35,683 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=22, state=SUCCESS; OpenRegionProcedure 2054d9ec275367d29389fbedb9c09867, server=jenkins-hbase4.apache.org,38293,1689746130512 in 230 msec 2023-07-19 05:55:35,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=17 2023-07-19 05:55:35,687 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45061,1689746121819 after splitting done 2023-07-19 05:55:35,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN in 390 msec 2023-07-19 05:55:35,687 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45061,1689746121819 from processing; numProcessing=0 2023-07-19 05:55:35,690 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45061,1689746121819, splitWal=true, meta=false in 5.0080 sec 2023-07-19 05:55:35,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:36,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-19 05:55:36,291 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-19 05:55:36,293 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-19 05:55:36,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.759sec 2023-07-19 05:55:36,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-19 05:55:36,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-19 05:55:36,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-19 05:55:36,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34645,1689746130485-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-19 05:55:36,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34645,1689746130485-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-19 05:55:36,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-19 05:55:36,331 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x146f1488 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:36,336 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@290eb8d4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:36,338 DEBUG [hconnection-0xe07f67f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:36,339 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41610, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:36,342 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-19 05:55:36,342 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x146f1488 to 127.0.0.1:65243 2023-07-19 05:55:36,342 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:36,344 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:34645 after: jenkins-hbase4.apache.org:34645 2023-07-19 05:55:36,344 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x34c9fd7d to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:36,348 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5cc5195, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:36,762 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-19 05:55:36,762 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-19 05:55:36,763 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-19 05:55:36,763 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-19 05:55:36,850 DEBUG [hconnection-0x5418b322-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:36,852 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41612, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:55:36,878 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689746136877 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:36,882 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication16897461368770x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:36,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689746136877-0x1017c2601c00023 connected Start Replication Server start 2023-07-19 05:55:36,886 INFO [Listener at localhost/45003] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:36,886 INFO [Listener at localhost/45003] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:36,901 INFO [Listener at localhost/45003] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-19 05:55:36,902 DEBUG [Listener at localhost/45003] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:36,912 INFO [Listener at localhost/45003] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:55:36,914 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,43759,1689746117439, jenkins-hbase4.apache.org,43215,1689746117482] other RSs: [] 2023-07-19 05:55:36,938 DEBUG [Listener at localhost/45003.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1ffcc516 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:36,945 DEBUG [Listener at localhost/45003.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5ada0522, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:36,945 INFO [Listener at localhost/45003.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:36,950 DEBUG [Listener at localhost/45003.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:36,951 INFO [Listener at localhost/45003.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:36,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c00025 connected 2023-07-19 05:55:40,075 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,43759,1689746117439/1's WALs to 1689746136886.syncuptool.replication.org,1234,1 2023-07-19 05:55:40,077 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 with data 2023-07-19 05:55:40,077 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109 with data PBUF\x08\xDFU 2023-07-19 05:55:40,082 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,43759,1689746117439/1's WALs to 1689746136886.syncuptool.replication.org,1234,1 2023-07-19 05:55:40,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:41,114 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:55:42,084 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43759,1689746117439] 2023-07-19 05:55:42,084 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43759,1689746117439] 2023-07-19 05:55:42,098 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43759,1689746117439] 2023-07-19 05:55:42,098 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,43759,1689746117439, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:55:42,099 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,118 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] zookeeper.ReadOnlyZKClient(139): Connect 0x67e04b10 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:42,125 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@29205505, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:42,125 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:42,129 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:42,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c00027 connected 2023-07-19 05:55:42,130 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,43759,1689746117439 (queues=2) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:42,130 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1 2023-07-19 05:55:42,134 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,43759,1689746117439, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:42,134 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0 2023-07-19 05:55:42,134 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1 2023-07-19 05:55:42,135 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,43759,1689746117439, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:42,136 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0 2023-07-19 05:55:42,142 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109 2023-07-19 05:55:42,142 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-19 05:55:42,142 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-19 05:55:42,143 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43759,1689746117439 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-19 05:55:42,145 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 2023-07-19 05:55:42,145 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-19 05:55:42,145 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43759,1689746117439 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:42,146 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:42,146 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,152 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41770, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:42,153 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38293] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:42,153 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38293] zookeeper.ReadOnlyZKClient(139): Connect 0x5616a327 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:42,159 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38293] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@58d15795, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:42,160 DEBUG [hconnection-0x5ceec857-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:55:42,162 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41778, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-19 05:55:42,168 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38293] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:42,169 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43759,1689746117439] 2023-07-19 05:55:42,170 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1.1689746124109] 2023-07-19 05:55:42,172 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43759,1689746117439 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:42,172 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,247 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,248 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] zookeeper.ZKUtil(162): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 2023-07-19 05:55:42,249 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43759,1689746117439/jenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0.1689746118997 2023-07-19 05:55:42,250 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,250 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,250 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-0,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,43759,1689746117439 with the following stats: Total replicated edits: 1, current progress: 2023-07-19 05:55:42,350 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,43759,1689746117439 2023-07-19 05:55:42,351 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] zookeeper.ZKUtil(243): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43759,1689746117439 because node does not exist (not an error) 2023-07-19 05:55:42,351 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439.replicationSource.shipperjenkins-hbase4.apache.org%2C43759%2C1689746117439.jenkins-hbase4.apache.org%2C43759%2C1689746117439.regiongroup-1,1-jenkins-hbase4.apache.org,43759,1689746117439] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,43759,1689746117439 with the following stats: Total replicated edits: 1, current progress: 2023-07-19 05:55:45,735 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,43215,1689746117482/1's WALs to 1689746136886.syncuptool.replication.org,1234,1 2023-07-19 05:55:45,738 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008 with data PBUF\x08\xCD\x06 2023-07-19 05:55:45,738 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 with data PBUF\x08\xF8\xA9\x01 2023-07-19 05:55:45,740 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,43215,1689746117482/1's WALs to 1689746136886.syncuptool.replication.org,1234,1 2023-07-19 05:55:45,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:45,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 1 2023-07-19 05:55:47,742 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:47,742 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:47,750 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:55:47,753 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:47,753 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,43215,1689746117482, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:55:47,753 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:47,774 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] zookeeper.ReadOnlyZKClient(139): Connect 0x13ba4784 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:47,779 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5fb1428d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:47,779 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:47,782 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:47,783 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,43215,1689746117482 (queues=2) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:47,783 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1 2023-07-19 05:55:47,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c0002a connected 2023-07-19 05:55:47,787 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,43215,1689746117482, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:47,787 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0 2023-07-19 05:55:47,787 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1 2023-07-19 05:55:47,790 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,43215,1689746117482, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:47,797 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0 2023-07-19 05:55:47,801 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008 2023-07-19 05:55:47,801 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:55:47,802 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-19 05:55:47,802 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43215,1689746117482 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-07-19 05:55:47,803 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:47,804 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997 2023-07-19 05:55:47,804 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-19 05:55:47,805 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-19 05:55:47,806 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43215,1689746117482 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-19 05:55:47,806 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1.1689746120008] 2023-07-19 05:55:47,808 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43215,1689746117482 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:47,808 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:47,809 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:47,811 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41792, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:47,812 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38293] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:55:47,821 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38293] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:55:47,821 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,43215,1689746117482] 2023-07-19 05:55:47,825 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0.1689746118997] 2023-07-19 05:55:47,827 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,43215,1689746117482 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:55:47,827 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:47,908 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:47,910 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:47,910 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-1,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,43215,1689746117482 with the following stats: Total replicated edits: 1, current progress: 2023-07-19 05:55:48,010 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,43215,1689746117482 2023-07-19 05:55:48,011 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] zookeeper.ZKUtil(243): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689746136886.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,43215,1689746117482 because node does not exist (not an error) 2023-07-19 05:55:48,011 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482.replicationSource.shipperjenkins-hbase4.apache.org%2C43215%2C1689746117482.jenkins-hbase4.apache.org%2C43215%2C1689746117482.regiongroup-0,1-jenkins-hbase4.apache.org,43215,1689746117482] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,43215,1689746117482 with the following stats: Total replicated edits: 1, current progress: 2023-07-19 05:55:50,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:50,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-19 05:55:53,411 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:55:55,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:55:55,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-19 05:55:56,915 INFO [Listener at localhost/45003] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-19 05:55:57,019 DEBUG [Listener at localhost/45003.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00025, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:57,019 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1ffcc516 to 127.0.0.1:65243 2023-07-19 05:55:57,020 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:55:57,222 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:57,222 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746136877-0x1017c2601c00023, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:55:57,232 INFO [Listener at localhost/45003] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-19 05:55:57,232 DEBUG [Listener at localhost/45003] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-19 05:55:57,233 INFO [Listener at localhost/45003] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-19 05:55:57,233 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:57,233 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:57,234 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:57,234 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:57,234 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:57,234 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:57,235 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38455 2023-07-19 05:55:57,235 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:57,236 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:57,237 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38455 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:57,239 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:384550x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:57,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38455-0x1017c2601c0002b connected 2023-07-19 05:55:57,243 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:55:57,243 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:57,244 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-19 05:55:57,244 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38455 2023-07-19 05:55:57,247 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38455 2023-07-19 05:55:57,250 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38455 2023-07-19 05:55:57,251 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38455 2023-07-19 05:55:57,251 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38455 2023-07-19 05:55:57,251 INFO [Listener at localhost/45003] master.HMaster(444): hbase.rootdir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320, hbase.cluster.distributed=false 2023-07-19 05:55:57,252 DEBUG [pool-566-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-19 05:55:57,262 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-19 05:55:57,262 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:57,263 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:57,263 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:55:57,263 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:55:57,263 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:55:57,263 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:55:57,264 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42035 2023-07-19 05:55:57,264 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:55:57,265 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:55:57,266 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:57,267 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:57,268 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42035 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:57,270 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:420350x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:57,272 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:420350x0, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:55:57,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42035-0x1017c2601c0002c connected 2023-07-19 05:55:57,273 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:55:57,273 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-19 05:55:57,274 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42035 2023-07-19 05:55:57,274 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42035 2023-07-19 05:55:57,276 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42035 2023-07-19 05:55:57,277 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42035 2023-07-19 05:55:57,278 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42035 2023-07-19 05:55:57,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,282 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-19 05:55:57,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,284 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-19 05:55:57,284 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-19 05:55:57,284 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:57,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-19 05:55:57,286 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,38455,1689746157233 from backup master directory 2023-07-19 05:55:57,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-19 05:55:57,292 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,292 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-19 05:55:57,292 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:57,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:55:57,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2d9aa771 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:57,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@526eb823, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:57,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:55:57,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-19 05:55:57,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:57,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:57,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265 to hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265-dead as it is dead 2023-07-19 05:55:57,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265-dead/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 2023-07-19 05:55:57,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265-dead/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 after 1ms 2023-07-19 05:55:57,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265-dead/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 to hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 2023-07-19 05:55:57,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,39519,1689746116265-dead 2023-07-19 05:55:57,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:57,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38455%2C1689746157233.jenkins-hbase4.apache.org%2C38455%2C1689746157233.regiongroup-0, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,38455,1689746157233, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/oldWALs, maxLogs=10 2023-07-19 05:55:57,359 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:57,359 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:57,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,38455,1689746157233/jenkins-hbase4.apache.org%2C38455%2C1689746157233.jenkins-hbase4.apache.org%2C38455%2C1689746157233.regiongroup-0.1689746157347 2023-07-19 05:55:57,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:55:57,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:57,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:57,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:57,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:57,365 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:57,366 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-19 05:55:57,366 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-19 05:55:57,374 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/25c9c9ede8c1485a853e23ea9291ba92 2023-07-19 05:55:57,375 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:57,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-19 05:55:57,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 2023-07-19 05:55:57,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 2023-07-19 05:55:57,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39519%2C1689746116265.jenkins-hbase4.apache.org%2C39519%2C1689746116265.regiongroup-0.1689746117913 2023-07-19 05:55:57,383 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:55:57,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-19 05:55:57,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59208444, jitterRate=-0.1177254319190979}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:55:57,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:55:57,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-19 05:55:57,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-19 05:55:57,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-19 05:55:57,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-19 05:55:57,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-19 05:55:57,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-19 05:55:57,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-19 05:55:57,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-19 05:55:57,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-19 05:55:57,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-19 05:55:57,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-19 05:55:57,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-19 05:55:57,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-19 05:55:57,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-19 05:55:57,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-19 05:55:57,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43759,1689746117439, table=hbase:meta, region=1588230740 2023-07-19 05:55:57,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-19 05:55:57,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,43215,1689746117482 already deleted, retry=false 2023-07-19 05:55:57,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43215,1689746117482 on jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43215,1689746117482, splitWal=true, meta=false 2023-07-19 05:55:57,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,43215,1689746117482 (carryingMeta=false) jenkins-hbase4.apache.org,43215,1689746117482/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@39968d3e[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:55:57,403 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,43759,1689746117439 already deleted, retry=false 2023-07-19 05:55:57,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43759,1689746117439 on jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43759,1689746117439, splitWal=true, meta=true 2023-07-19 05:55:57,404 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,43759,1689746117439 (carryingMeta=true) jenkins-hbase4.apache.org,43759,1689746117439/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@4d351701[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:55:57,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-19 05:55:57,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-19 05:55:57,415 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-19 05:55:57,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-19 05:55:57,415 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-19 05:55:57,416 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-19 05:55:57,416 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-19 05:55:57,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-19 05:55:57,420 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-19 05:55:57,420 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-19 05:55:57,420 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:55:57,421 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38455,1689746157233, sessionid=0x1017c2601c0002b, setting cluster-up flag (Was=false) 2023-07-19 05:55:57,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-19 05:55:57,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-19 05:55:57,432 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:55:57,433 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/.hbase-snapshot/.tmp 2023-07-19 05:55:57,436 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:57,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689746187443 2023-07-19 05:55:57,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-19 05:55:57,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-19 05:55:57,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-19 05:55:57,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-19 05:55:57,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-19 05:55:57,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-19 05:55:57,445 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43759,1689746117439; numProcessing=1 2023-07-19 05:55:57,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,445 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43215,1689746117482; numProcessing=2 2023-07-19 05:55:57,445 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43759,1689746117439, splitWal=true, meta=true 2023-07-19 05:55:57,445 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43215,1689746117482, splitWal=true, meta=false 2023-07-19 05:55:57,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-19 05:55:57,447 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43759,1689746117439, splitWal=true, meta=true, isMeta: true 2023-07-19 05:55:57,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-19 05:55:57,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-19 05:55:57,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-19 05:55:57,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-19 05:55:57,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746157449,5,FailOnTimeoutGroup] 2023-07-19 05:55:57,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746157449,5,FailOnTimeoutGroup] 2023-07-19 05:55:57,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-19 05:55:57,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689746157449, completionTime=-1 2023-07-19 05:55:57,449 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439-splitting 2023-07-19 05:55:57,449 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-19 05:55:57,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-19 05:55:57,450 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439-splitting dir is empty, no logs to split. 2023-07-19 05:55:57,450 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,43759,1689746117439 WAL count=0, meta=true 2023-07-19 05:55:57,452 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439-splitting dir is empty, no logs to split. 2023-07-19 05:55:57,452 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,43759,1689746117439 WAL count=0, meta=true 2023-07-19 05:55:57,452 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43759,1689746117439 WAL splitting is done? wals=0, meta=true 2023-07-19 05:55:57,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-19 05:55:57,454 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-19 05:55:57,455 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:55:57,480 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(951): ClusterId : fa474148-f7f6-4a32-a2c9-9f3f924e734e 2023-07-19 05:55:57,480 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:55:57,482 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:55:57,482 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:55:57,484 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:55:57,486 DEBUG [RS:0;jenkins-hbase4:42035] zookeeper.ReadOnlyZKClient(139): Connect 0x6add172b to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:57,489 DEBUG [RS:0;jenkins-hbase4:42035] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@641ae400, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:57,489 DEBUG [RS:0;jenkins-hbase4:42035] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60257c5b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:55:57,497 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42035 2023-07-19 05:55:57,498 INFO [RS:0;jenkins-hbase4:42035] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:55:57,498 INFO [RS:0;jenkins-hbase4:42035] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:55:57,498 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:55:57,498 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38455,1689746157233 with isa=jenkins-hbase4.apache.org/172.31.14.131:42035, startcode=1689746157262 2023-07-19 05:55:57,498 DEBUG [RS:0;jenkins-hbase4:42035] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:55:57,500 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53519, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:55:57,501 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38455] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,501 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320 2023-07-19 05:55:57,501 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46083 2023-07-19 05:55:57,501 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:55:57,503 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:55:57,504 DEBUG [RS:0;jenkins-hbase4:42035] zookeeper.ZKUtil(162): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,504 WARN [RS:0;jenkins-hbase4:42035] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:55:57,504 INFO [RS:0;jenkins-hbase4:42035] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:57,505 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:55:57,505 INFO [RS:0;jenkins-hbase4:42035] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:57,505 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,520 INFO [RS:0;jenkins-hbase4:42035] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-19 05:55:57,520 DEBUG [RS:0;jenkins-hbase4:42035] zookeeper.ZKUtil(162): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,521 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:55:57,521 INFO [RS:0;jenkins-hbase4:42035] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:55:57,522 INFO [RS:0;jenkins-hbase4:42035] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:55:57,523 INFO [RS:0;jenkins-hbase4:42035] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:55:57,523 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,523 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:55:57,524 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,525 DEBUG [RS:0;jenkins-hbase4:42035] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:55:57,526 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,526 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,527 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,537 INFO [RS:0;jenkins-hbase4:42035] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:55:57,537 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42035,1689746157262-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:55:57,547 INFO [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:55:57,548 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689746136886.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:55:57,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-19 05:55:57,560 INFO [RS:0;jenkins-hbase4:42035] regionserver.Replication(203): jenkins-hbase4.apache.org,42035,1689746157262 started 2023-07-19 05:55:57,560 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42035,1689746157262, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42035, sessionid=0x1017c2601c0002c 2023-07-19 05:55:57,560 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:55:57,560 DEBUG [RS:0;jenkins-hbase4:42035] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,560 DEBUG [RS:0;jenkins-hbase4:42035] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42035,1689746157262' 2023-07-19 05:55:57,560 DEBUG [RS:0;jenkins-hbase4:42035] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42035,1689746157262' 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-19 05:55:57,561 DEBUG [RS:0;jenkins-hbase4:42035] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-19 05:55:57,562 DEBUG [RS:0;jenkins-hbase4:42035] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:55:57,562 INFO [RS:0;jenkins-hbase4:42035] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:55:57,562 INFO [RS:0;jenkins-hbase4:42035] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:55:57,567 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x0bb6bc80 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:55:57,570 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@359daecc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:55:57,571 INFO [RS:0;jenkins-hbase4:42035.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:55:57,573 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:55:57,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c00030 connected 2023-07-19 05:55:57,576 INFO [RS:0;jenkins-hbase4:42035.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:55:57,606 DEBUG [jenkins-hbase4:38455] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-19 05:55:57,607 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42035,1689746157262, state=OPENING 2023-07-19 05:55:57,608 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-19 05:55:57,608 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-19 05:55:57,608 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42035,1689746157262}] 2023-07-19 05:55:57,662 INFO [RS:0;jenkins-hbase4:42035] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:57,664 INFO [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:57,678 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:57,678 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:57,690 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0 for peer 1 2023-07-19 05:55:57,690 INFO [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 2023-07-19 05:55:57,690 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0 2023-07-19 05:55:57,690 INFO [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:55:57,690 DEBUG [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:55:57,690 INFO [RS:0;jenkins-hbase4:42035.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0 2023-07-19 05:55:57,695 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:57,765 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:55:57,766 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:55:57,767 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54818, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:55:57,770 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-19 05:55:57,770 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:55:57,770 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:55:57,770 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:55:57,772 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42035%2C1689746157262.meta, suffix=.meta, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:55:57,784 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:55:57,784 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:55:57,786 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.meta.1689746157772.meta 2023-07-19 05:55:57,786 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:55:57,786 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:55:57,786 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-19 05:55:57,786 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-19 05:55:57,787 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-19 05:55:57,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-19 05:55:57,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:55:57,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-19 05:55:57,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-19 05:55:57,788 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:55:57,789 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info 2023-07-19 05:55:57,789 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info 2023-07-19 05:55:57,790 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:55:57,796 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info/ee0fa4692c5f40eca008529c87cbb8d9 2023-07-19 05:55:57,796 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:57,796 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:55:57,797 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:57,797 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:55:57,798 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:55:57,799 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:57,803 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier/44f4e872a3654b3e9e31e378b714b20c 2023-07-19 05:55:57,803 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:57,803 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:55:57,804 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table 2023-07-19 05:55:57,804 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table 2023-07-19 05:55:57,804 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:55:57,809 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/table/ddf6019c76084b638ea6c99ae82d5c5f 2023-07-19 05:55:57,809 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:55:57,810 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740 2023-07-19 05:55:57,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740 2023-07-19 05:55:57,813 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:55:57,814 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:55:57,815 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62989483, jitterRate=-0.06138356029987335}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:55:57,815 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:55:57,816 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689746157765 2023-07-19 05:55:57,820 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-19 05:55:57,821 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-19 05:55:57,821 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42035,1689746157262, state=OPEN 2023-07-19 05:55:57,823 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-19 05:55:57,823 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-19 05:55:57,825 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-19 05:55:57,825 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42035,1689746157262 in 215 msec 2023-07-19 05:55:57,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-19 05:55:57,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 372 msec 2023-07-19 05:55:57,903 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,008 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,112 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,217 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,322 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,427 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,531 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,636 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,741 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,845 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:58,950 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-19 05:55:59,054 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,159 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,263 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,367 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,472 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,577 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,681 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,785 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,890 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:55:59,994 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,098 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,203 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,307 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,411 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,516 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-19 05:56:00,620 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,727 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:56:00,788 INFO [jenkins-hbase4:38293Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-19 05:56:00,832 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:00,936 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,041 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,146 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,251 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,355 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,460 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,564 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,669 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,773 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,878 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-19 05:56:01,956 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-19 05:56:01,960 DEBUG [hconnection-0xf7b9f3a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:01,961 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41648, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:56:01,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=13c92efbf45fed23cebac57cf655e861, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43215,1689746117482, regionLocation=jenkins-hbase4.apache.org,43215,1689746117482, openSeqNum=2 2023-07-19 05:56:01,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=8c7505042d76fb8c47639ac419e6a996, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43759,1689746117439, regionLocation=jenkins-hbase4.apache.org,43759,1689746117439, openSeqNum=2 2023-07-19 05:56:01,968 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=8f71d242d3ffe63977903bdc0b5b9fe2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43215,1689746117482, regionLocation=jenkins-hbase4.apache.org,43215,1689746117482, openSeqNum=2 2023-07-19 05:56:01,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-19 05:56:01,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689746221968 2023-07-19 05:56:01,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689746281968 2023-07-19 05:56:01,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-19 05:56:01,983 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:01,987 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43759,1689746117439 had 2 regions 2023-07-19 05:56:01,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38455,1689746157233-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:01,987 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43215,1689746117482 had 2 regions 2023-07-19 05:56:01,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38455,1689746157233-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:01,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38455,1689746157233-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:01,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38455, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:01,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:01,987 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. is NOT online; state={13c92efbf45fed23cebac57cf655e861 state=OPEN, ts=1689746161967, server=jenkins-hbase4.apache.org,43215,1689746117482}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-19 05:56:01,989 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43759,1689746117439, splitWal=true, meta=true, isMeta: false 2023-07-19 05:56:01,989 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43215,1689746117482, splitWal=true, meta=false, isMeta: false 2023-07-19 05:56:01,991 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439-splitting dir is empty, no logs to split. 2023-07-19 05:56:01,991 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,43759,1689746117439 WAL count=0, meta=false 2023-07-19 05:56:01,991 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,43215,1689746117482/hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861., unknown_server=jenkins-hbase4.apache.org,43759,1689746117439/t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996., unknown_server=jenkins-hbase4.apache.org,43215,1689746117482/t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:01,991 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482-splitting 2023-07-19 05:56:01,992 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482-splitting dir is empty, no logs to split. 2023-07-19 05:56:01,992 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,43215,1689746117482 WAL count=0, meta=false 2023-07-19 05:56:01,993 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43759,1689746117439-splitting dir is empty, no logs to split. 2023-07-19 05:56:01,994 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,43759,1689746117439 WAL count=0, meta=false 2023-07-19 05:56:01,994 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43759,1689746117439 WAL splitting is done? wals=0, meta=false 2023-07-19 05:56:01,995 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482-splitting dir is empty, no logs to split. 2023-07-19 05:56:01,995 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,43215,1689746117482 WAL count=0, meta=false 2023-07-19 05:56:01,995 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43215,1689746117482 WAL splitting is done? wals=0, meta=false 2023-07-19 05:56:01,995 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN}] 2023-07-19 05:56:01,996 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN 2023-07-19 05:56:01,996 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:01,997 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43215,1689746117482 failed, ignore...File hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,43215,1689746117482-splitting does not exist. 2023-07-19 05:56:01,998 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN}, {pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN}] 2023-07-19 05:56:01,998 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN 2023-07-19 05:56:01,999 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN 2023-07-19 05:56:01,999 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:01,999 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:01,999 DEBUG [jenkins-hbase4:38455] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-19 05:56:02,001 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=8c7505042d76fb8c47639ac419e6a996, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:02,002 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746162001"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746162001"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746162001"}]},"ts":"1689746162001"} 2023-07-19 05:56:02,002 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=13c92efbf45fed23cebac57cf655e861, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:02,002 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746162002"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746162002"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746162002"}]},"ts":"1689746162002"} 2023-07-19 05:56:02,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 8c7505042d76fb8c47639ac419e6a996, server=jenkins-hbase4.apache.org,42035,1689746157262}] 2023-07-19 05:56:02,005 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 13c92efbf45fed23cebac57cf655e861, server=jenkins-hbase4.apache.org,42035,1689746157262}] 2023-07-19 05:56:02,088 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:02,153 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=8f71d242d3ffe63977903bdc0b5b9fe2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:02,153 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746162153"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746162153"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746162153"}]},"ts":"1689746162153"} 2023-07-19 05:56:02,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure 8f71d242d3ffe63977903bdc0b5b9fe2, server=jenkins-hbase4.apache.org,42035,1689746157262}] 2023-07-19 05:56:02,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:02,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:56:02,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1, suffix=, logDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262, archiveDir=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs, maxLogs=10 2023-07-19 05:56:02,162 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:02,162 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 13c92efbf45fed23cebac57cf655e861, NAME => 'hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:02,162 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,162 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:02,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,164 INFO [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,165 DEBUG [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info 2023-07-19 05:56:02,165 DEBUG [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info 2023-07-19 05:56:02,165 INFO [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 13c92efbf45fed23cebac57cf655e861 columnFamilyName info 2023-07-19 05:56:02,171 DEBUG [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/info/291551e7f9204b53bba847f884f25d46 2023-07-19 05:56:02,171 INFO [StoreOpener-13c92efbf45fed23cebac57cf655e861-1] regionserver.HStore(310): Store=13c92efbf45fed23cebac57cf655e861/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:02,172 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,176 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:02,177 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 13c92efbf45fed23cebac57cf655e861; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70898041, jitterRate=0.05646313726902008}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:56:02,177 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 13c92efbf45fed23cebac57cf655e861: 2023-07-19 05:56:02,177 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861., pid=26, masterSystemTime=1689746162156 2023-07-19 05:56:02,181 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:02,181 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:02,181 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=13c92efbf45fed23cebac57cf655e861, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:02,181 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK] 2023-07-19 05:56:02,181 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746162181"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746162181"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746162181"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746162181"}]},"ts":"1689746162181"} 2023-07-19 05:56:02,181 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK] 2023-07-19 05:56:02,185 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-19 05:56:02,186 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 13c92efbf45fed23cebac57cf655e861, server=jenkins-hbase4.apache.org,42035,1689746157262 in 178 msec 2023-07-19 05:56:02,187 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=13c92efbf45fed23cebac57cf655e861, ASSIGN in 187 msec 2023-07-19 05:56:02,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1 for peer 1 2023-07-19 05:56:02,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162 2023-07-19 05:56:02,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1 2023-07-19 05:56:02,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:56:02,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41111,DS-5a67531b-e36f-4c92-88c0-75ac8af65e90,DISK], DatanodeInfoWithStorage[127.0.0.1:33877,DS-0c2e7b02-5e44-4d7f-8836-231fbb097a30,DISK]] 2023-07-19 05:56:02,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1 2023-07-19 05:56:02,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8c7505042d76fb8c47639ac419e6a996, NAME => 't1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:02,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:02,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,193 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,193 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,193 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:56:02,193 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,193 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:56:02,194 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1 2023-07-19 05:56:02,194 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1 2023-07-19 05:56:02,195 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c7505042d76fb8c47639ac419e6a996 columnFamilyName cf1 2023-07-19 05:56:02,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-19 05:56:02,202 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8755977c20064a56a422f0f1b8af659c 2023-07-19 05:56:02,202 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1/8755977c20064a56a422f0f1b8af659c 2023-07-19 05:56:02,202 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(310): Store=8c7505042d76fb8c47639ac419e6a996/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:02,202 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,203 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep 2023-07-19 05:56:02,203 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep 2023-07-19 05:56:02,204 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c7505042d76fb8c47639ac419e6a996 columnFamilyName norep 2023-07-19 05:56:02,210 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1214f5f573c94782b0b51c9f3fc07563 2023-07-19 05:56:02,210 DEBUG [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep/1214f5f573c94782b0b51c9f3fc07563 2023-07-19 05:56:02,210 INFO [StoreOpener-8c7505042d76fb8c47639ac419e6a996-1] regionserver.HStore(310): Store=8c7505042d76fb8c47639ac419e6a996/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:02,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:56:02,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:02,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8c7505042d76fb8c47639ac419e6a996; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71776691, jitterRate=0.06955604255199432}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:56:02,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8c7505042d76fb8c47639ac419e6a996: 2023-07-19 05:56:02,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996., pid=25, masterSystemTime=1689746162156 2023-07-19 05:56:02,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:02,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:02,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=8c7505042d76fb8c47639ac419e6a996, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:02,219 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746162218"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746162218"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746162218"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746162218"}]},"ts":"1689746162218"} 2023-07-19 05:56:02,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-19 05:56:02,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 8c7505042d76fb8c47639ac419e6a996, server=jenkins-hbase4.apache.org,42035,1689746157262 in 216 msec 2023-07-19 05:56:02,223 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=19 2023-07-19 05:56:02,223 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43759,1689746117439 after splitting done 2023-07-19 05:56:02,223 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=8c7505042d76fb8c47639ac419e6a996, ASSIGN in 227 msec 2023-07-19 05:56:02,223 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43759,1689746117439 from processing; numProcessing=1 2023-07-19 05:56:02,224 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43759,1689746117439, splitWal=true, meta=true in 4.8190 sec 2023-07-19 05:56:02,299 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-19 05:56:02,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:56:02,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-19 05:56:02,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162, lastWalPosition=452, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:56:02,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:02,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8f71d242d3ffe63977903bdc0b5b9fe2, NAME => 't2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:02,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:02,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,312 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,313 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1 2023-07-19 05:56:02,313 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1 2023-07-19 05:56:02,314 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f71d242d3ffe63977903bdc0b5b9fe2 columnFamilyName cf1 2023-07-19 05:56:02,320 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ed5951d0933423da886fa0a7efe3914 2023-07-19 05:56:02,320 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1/5ed5951d0933423da886fa0a7efe3914 2023-07-19 05:56:02,320 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(310): Store=8f71d242d3ffe63977903bdc0b5b9fe2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:02,320 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,321 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep 2023-07-19 05:56:02,321 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep 2023-07-19 05:56:02,321 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f71d242d3ffe63977903bdc0b5b9fe2 columnFamilyName norep 2023-07-19 05:56:02,327 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 938d1f6c4e354a2a94f0ac8447e5a2c3 2023-07-19 05:56:02,327 DEBUG [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(539): loaded hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep/938d1f6c4e354a2a94f0ac8447e5a2c3 2023-07-19 05:56:02,327 INFO [StoreOpener-8f71d242d3ffe63977903bdc0b5b9fe2-1] regionserver.HStore(310): Store=8f71d242d3ffe63977903bdc0b5b9fe2/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:02,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:56:02,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:02,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8f71d242d3ffe63977903bdc0b5b9fe2; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74992740, jitterRate=0.11747890710830688}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:56:02,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8f71d242d3ffe63977903bdc0b5b9fe2: 2023-07-19 05:56:02,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2., pid=27, masterSystemTime=1689746162307 2023-07-19 05:56:02,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:02,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:02,338 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=8f71d242d3ffe63977903bdc0b5b9fe2, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:02,338 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746162338"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746162338"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746162338"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746162338"}]},"ts":"1689746162338"} 2023-07-19 05:56:02,341 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-07-19 05:56:02,341 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure 8f71d242d3ffe63977903bdc0b5b9fe2, server=jenkins-hbase4.apache.org,42035,1689746157262 in 184 msec 2023-07-19 05:56:02,343 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=18 2023-07-19 05:56:02,343 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43215,1689746117482 after splitting done 2023-07-19 05:56:02,343 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=8f71d242d3ffe63977903bdc0b5b9fe2, ASSIGN in 343 msec 2023-07-19 05:56:02,343 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43215,1689746117482 from processing; numProcessing=0 2023-07-19 05:56:02,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43215,1689746117482, splitWal=true, meta=false in 4.9430 sec 2023-07-19 05:56:02,404 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:56:02,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162, lastWalPosition=823, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:56:02,509 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,524 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:56:02,570 INFO [jenkins-hbase4:42035Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0]: currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 at position: 422 walGroup [jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1]: currently replicating from: hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162 at position: 823 2023-07-19 05:56:02,615 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,720 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,824 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,929 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:02,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:02,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-19 05:56:03,001 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-19 05:56:03,004 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-19 05:56:03,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.712sec 2023-07-19 05:56:03,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-19 05:56:03,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-19 05:56:03,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-19 05:56:03,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38455,1689746157233-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-19 05:56:03,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38455,1689746157233-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-19 05:56:03,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-19 05:56:03,033 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,084 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x13eb5427 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:03,088 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43a3144c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:03,089 DEBUG [hconnection-0x23ccca19-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:03,091 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41656, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:56:03,096 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-19 05:56:03,096 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13eb5427 to 127.0.0.1:65243 2023-07-19 05:56:03,097 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,098 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:38455 after: jenkins-hbase4.apache.org:38455 2023-07-19 05:56:03,098 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x71a206a1 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:03,101 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@347b868e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:03,102 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34c9fd7d to 127.0.0.1:65243 2023-07-19 05:56:03,102 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,102 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-19 05:56:03,102 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(257): Found active master hash=171011777, stopped=false 2023-07-19 05:56:03,102 INFO [Listener at localhost/45003] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:56:03,103 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:56:03,103 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:56:03,104 INFO [Listener at localhost/45003] procedure2.ProcedureExecutor(629): Stopping 2023-07-19 05:56:03,104 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:56:03,104 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x60e35b4e to 127.0.0.1:65243 2023-07-19 05:56:03,105 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:56:03,105 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:56:03,106 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38293,1689746130512' ***** 2023-07-19 05:56:03,106 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:56:03,106 INFO [RS:0;jenkins-hbase4:38293] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:56:03,107 INFO [RS:0;jenkins-hbase4:38293] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:56:03,107 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:56:03,107 INFO [RS:0;jenkins-hbase4:38293] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(3305): Received CLOSE for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(3305): Received CLOSE for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(3305): Received CLOSE for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:56:03,108 DEBUG [RS:0;jenkins-hbase4:38293] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x113728db to 127.0.0.1:65243 2023-07-19 05:56:03,108 DEBUG [RS:0;jenkins-hbase4:38293] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:56:03,108 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-19 05:56:03,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 168d6c8e660cdb03a471b9cc5e7a3e2b, disabling compactions & flushes 2023-07-19 05:56:03,111 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-19 05:56:03,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:03,112 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1478): Online Regions={168d6c8e660cdb03a471b9cc5e7a3e2b=t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b., ecfca84342d05307f433bda44290dd5a=hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., 1588230740=hbase:meta,,1.1588230740, 2054d9ec275367d29389fbedb9c09867=t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.} 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:56:03,112 DEBUG [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1504): Waiting on 1588230740, 168d6c8e660cdb03a471b9cc5e7a3e2b, 2054d9ec275367d29389fbedb9c09867, ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:03,112 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. after waiting 0 ms 2023-07-19 05:56:03,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:03,112 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-19 05:56:03,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 168d6c8e660cdb03a471b9cc5e7a3e2b 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-19 05:56:03,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=16 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/.tmp/cf1/2ea0ffeb6d72499d849182a16f9441a7 2023-07-19 05:56:03,134 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/info/fe9f1349700b4d27bc86accbfe86fb7c 2023-07-19 05:56:03,138 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ea0ffeb6d72499d849182a16f9441a7 2023-07-19 05:56:03,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/.tmp/cf1/2ea0ffeb6d72499d849182a16f9441a7 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/2ea0ffeb6d72499d849182a16f9441a7 2023-07-19 05:56:03,142 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/info/fe9f1349700b4d27bc86accbfe86fb7c as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/fe9f1349700b4d27bc86accbfe86fb7c 2023-07-19 05:56:03,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ea0ffeb6d72499d849182a16f9441a7 2023-07-19 05:56:03,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/2ea0ffeb6d72499d849182a16f9441a7, entries=100, sequenceid=16, filesize=7.9 K 2023-07-19 05:56:03,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 168d6c8e660cdb03a471b9cc5e7a3e2b in 35ms, sequenceid=16, compaction requested=false 2023-07-19 05:56:03,147 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/fe9f1349700b4d27bc86accbfe86fb7c, entries=24, sequenceid=31, filesize=7.4 K 2023-07-19 05:56:03,149 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 37ms, sequenceid=31, compaction requested=false 2023-07-19 05:56:03,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/recovered.edits/19.seqid, newMaxSeqId=19, maxSeqId=12 2023-07-19 05:56:03,166 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-19 05:56:03,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:03,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:56:03,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:03,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ecfca84342d05307f433bda44290dd5a, disabling compactions & flushes 2023-07-19 05:56:03,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:03,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:03,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. after waiting 0 ms 2023-07-19 05:56:03,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:03,168 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-19 05:56:03,169 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:56:03,169 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:56:03,169 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-19 05:56:03,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-19 05:56:03,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:03,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:56:03,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:03,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2054d9ec275367d29389fbedb9c09867, disabling compactions & flushes 2023-07-19 05:56:03,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:03,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:03,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. after waiting 0 ms 2023-07-19 05:56:03,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:03,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2054d9ec275367d29389fbedb9c09867 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-19 05:56:03,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/.tmp/cf1/281b662bf8104a34817e5c08d82ce4a0 2023-07-19 05:56:03,190 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:56:03,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 281b662bf8104a34817e5c08d82ce4a0 2023-07-19 05:56:03,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/.tmp/cf1/281b662bf8104a34817e5c08d82ce4a0 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/281b662bf8104a34817e5c08d82ce4a0 2023-07-19 05:56:03,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 281b662bf8104a34817e5c08d82ce4a0 2023-07-19 05:56:03,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/281b662bf8104a34817e5c08d82ce4a0, entries=50, sequenceid=14, filesize=6.4 K 2023-07-19 05:56:03,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 2054d9ec275367d29389fbedb9c09867 in 28ms, sequenceid=14, compaction requested=false 2023-07-19 05:56:03,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-19 05:56:03,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:03,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:56:03,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:03,242 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,312 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38293,1689746130512; all regions closed. 2023-07-19 05:56:03,317 DEBUG [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:56:03,317 INFO [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38293%2C1689746130512.meta:.meta(num 1689746131047) 2023-07-19 05:56:03,322 DEBUG [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:56:03,322 INFO [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38293%2C1689746130512.jenkins-hbase4.apache.org%2C38293%2C1689746130512.regiongroup-1:(num 1689746135608) 2023-07-19 05:56:03,328 DEBUG [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:56:03,328 INFO [RS:0;jenkins-hbase4:38293] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38293%2C1689746130512.jenkins-hbase4.apache.org%2C38293%2C1689746130512.regiongroup-0:(num 1689746130891) 2023-07-19 05:56:03,328 DEBUG [RS:0;jenkins-hbase4:38293] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,328 INFO [RS:0;jenkins-hbase4:38293] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:56:03,328 INFO [RS:0;jenkins-hbase4:38293] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-19 05:56:03,328 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:56:03,329 DEBUG [RS:0;jenkins-hbase4:38293] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5616a327 to 127.0.0.1:65243 2023-07-19 05:56:03,329 DEBUG [RS:0;jenkins-hbase4:38293] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,332 INFO [RS:0;jenkins-hbase4:38293] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38293 2023-07-19 05:56:03,336 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:03,336 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00027, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:03,336 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,38293,1689746130512 2023-07-19 05:56:03,336 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c0002a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:03,336 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:03,336 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:03,336 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:03,336 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38293,1689746130512] 2023-07-19 05:56:03,336 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38293,1689746130512; numProcessing=1 2023-07-19 05:56:03,338 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38293,1689746130512 already deleted, retry=false 2023-07-19 05:56:03,338 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38293,1689746130512 expired; onlineServers=0 2023-07-19 05:56:03,338 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34645,1689746130485' ***** 2023-07-19 05:56:03,338 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-19 05:56:03,339 DEBUG [M:0;jenkins-hbase4:34645] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e0eeb5f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:56:03,339 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34645,1689746130485 2023-07-19 05:56:03,339 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34645,1689746130485; all regions closed. 2023-07-19 05:56:03,339 DEBUG [M:0;jenkins-hbase4:34645] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:03,340 DEBUG [M:0;jenkins-hbase4:34645] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-19 05:56:03,340 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-19 05:56:03,340 DEBUG [M:0;jenkins-hbase4:34645] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-19 05:56:03,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746130717] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746130717,5,FailOnTimeoutGroup] 2023-07-19 05:56:03,340 INFO [M:0;jenkins-hbase4:34645] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-19 05:56:03,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746130716] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746130716,5,FailOnTimeoutGroup] 2023-07-19 05:56:03,341 INFO [M:0;jenkins-hbase4:34645] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-19 05:56:03,342 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-19 05:56:03,342 INFO [M:0;jenkins-hbase4:34645] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-19 05:56:03,342 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:56:03,342 DEBUG [M:0;jenkins-hbase4:34645] master.HMaster(1512): Stopping service threads 2023-07-19 05:56:03,342 INFO [M:0;jenkins-hbase4:34645] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-19 05:56:03,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:56:03,343 ERROR [M:0;jenkins-hbase4:34645] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-19 05:56:03,343 INFO [M:0;jenkins-hbase4:34645] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-19 05:56:03,343 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-19 05:56:03,344 DEBUG [M:0;jenkins-hbase4:34645] zookeeper.ZKUtil(398): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-19 05:56:03,344 WARN [M:0;jenkins-hbase4:34645] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-19 05:56:03,344 INFO [M:0;jenkins-hbase4:34645] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-19 05:56:03,345 INFO [M:0;jenkins-hbase4:34645] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-19 05:56:03,346 DEBUG [M:0;jenkins-hbase4:34645] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:56:03,346 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:03,346 DEBUG [M:0;jenkins-hbase4:34645] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:03,346 DEBUG [M:0;jenkins-hbase4:34645] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:56:03,346 DEBUG [M:0;jenkins-hbase4:34645] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:03,346 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.70 KB heapSize=47.15 KB 2023-07-19 05:56:03,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,353 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,364 INFO [M:0;jenkins-hbase4:34645] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.70 KB at sequenceid=239 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/23a3c3ef4a3e4624823144fba98d1ab1 2023-07-19 05:56:03,368 INFO [M:0;jenkins-hbase4:34645] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 23a3c3ef4a3e4624823144fba98d1ab1 2023-07-19 05:56:03,369 DEBUG [M:0;jenkins-hbase4:34645] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/23a3c3ef4a3e4624823144fba98d1ab1 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/23a3c3ef4a3e4624823144fba98d1ab1 2023-07-19 05:56:03,375 INFO [M:0;jenkins-hbase4:34645] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 23a3c3ef4a3e4624823144fba98d1ab1 2023-07-19 05:56:03,375 INFO [M:0;jenkins-hbase4:34645] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/23a3c3ef4a3e4624823144fba98d1ab1, entries=20, sequenceid=239, filesize=5.6 K 2023-07-19 05:56:03,376 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegion(2948): Finished flush of dataSize ~36.70 KB/37578, heapSize ~47.13 KB/48264, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=239, compaction requested=false 2023-07-19 05:56:03,378 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:03,378 DEBUG [M:0;jenkins-hbase4:34645] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:56:03,383 INFO [M:0;jenkins-hbase4:34645] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-19 05:56:03,383 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:56:03,384 INFO [M:0;jenkins-hbase4:34645] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34645 2023-07-19 05:56:03,388 DEBUG [M:0;jenkins-hbase4:34645] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,34645,1689746130485 already deleted, retry=false 2023-07-19 05:56:03,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,457 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,568 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,601 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:03,601 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:34645-0x1017c2601c0001d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:03,601 INFO [M:0;jenkins-hbase4:34645] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34645,1689746130485; zookeeper connection closed. 2023-07-19 05:56:03,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-19 05:56:03,672 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,701 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:03,701 INFO [RS:0;jenkins-hbase4:38293] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38293,1689746130512; zookeeper connection closed. 2023-07-19 05:56:03,701 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:38293-0x1017c2601c0001e, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:03,703 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@32c3eb4c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@32c3eb4c 2023-07-19 05:56:03,703 INFO [Listener at localhost/45003] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-19 05:56:03,705 DEBUG [hconnection-0xd5ec03f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:03,707 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:56:03,776 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6617 2023-07-19 05:56:03,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 55 WAL entries eligible for replication 2023-07-19 05:56:03,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6617 2023-07-19 05:56:03,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/112=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/113=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/114=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/115=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/116=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/117=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/118=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/119=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/120=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/121=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/122=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/123=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/124=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/125=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/126=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/127=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/128=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/129=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/130=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/131=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/132=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/133=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/134=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/135=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/136=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/137=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/138=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/139=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/140=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/141=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/142=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/143=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/144=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/145=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/146=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/147=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/148=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/149=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/150=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/151=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/152=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/153=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/154=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/155=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/156=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/157=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/158=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/159=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/160=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/161=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/162=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/163=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/164=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/165=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/166=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162, lastWalPosition=6617, nbRowKeys=55, nbHFiles=0, heapSize=11440, lastSeqIds={}, endOfFile=false,usedBufferSize=11440] 2023-07-19 05:56:03,781 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-19 05:56:03,882 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-19 05:56:03,882 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:03,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 16673 2023-07-19 05:56:03,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 94 WAL entries eligible for replication 2023-07-19 05:56:03,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 16673 2023-07-19 05:56:03,988 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:04,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 31043 2023-07-19 05:56:04,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 134 WAL entries eligible for replication 2023-07-19 05:56:04,037 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71a206a1 to 127.0.0.1:65243 2023-07-19 05:56:04,038 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:04,038 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-19 05:56:04,038 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(257): Found active master hash=290757171, stopped=false 2023-07-19 05:56:04,038 INFO [Listener at localhost/45003] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:56:04,040 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-19 05:56:04,040 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-19 05:56:04,040 INFO [Listener at localhost/45003] procedure2.ProcedureExecutor(629): Stopping 2023-07-19 05:56:04,040 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:56:04,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:56:04,043 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d9aa771 to 127.0.0.1:65243 2023-07-19 05:56:04,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-19 05:56:04,043 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:04,043 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42035,1689746157262' ***** 2023-07-19 05:56:04,043 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:56:04,043 INFO [RS:0;jenkins-hbase4:42035] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:56:04,044 INFO [RS:0;jenkins-hbase4:42035] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:56:04,044 INFO [RS:0;jenkins-hbase4:42035] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:56:04,044 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:56:04,044 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(3305): Received CLOSE for 8c7505042d76fb8c47639ac419e6a996 2023-07-19 05:56:04,045 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(3305): Received CLOSE for 13c92efbf45fed23cebac57cf655e861 2023-07-19 05:56:04,045 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(3305): Received CLOSE for 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:04,045 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:04,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8c7505042d76fb8c47639ac419e6a996, disabling compactions & flushes 2023-07-19 05:56:04,045 DEBUG [RS:0;jenkins-hbase4:42035] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6add172b to 127.0.0.1:65243 2023-07-19 05:56:04,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:04,045 DEBUG [RS:0;jenkins-hbase4:42035] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:04,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:04,045 INFO [RS:0;jenkins-hbase4:42035] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:56:04,045 INFO [RS:0;jenkins-hbase4:42035] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:56:04,045 INFO [RS:0;jenkins-hbase4:42035] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:56:04,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. after waiting 0 ms 2023-07-19 05:56:04,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:04,046 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-19 05:56:04,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8c7505042d76fb8c47639ac419e6a996 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-19 05:56:04,046 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-19 05:56:04,046 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1478): Online Regions={8c7505042d76fb8c47639ac419e6a996=t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996., 13c92efbf45fed23cebac57cf655e861=hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861., 1588230740=hbase:meta,,1.1588230740, 8f71d242d3ffe63977903bdc0b5b9fe2=t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2.} 2023-07-19 05:56:04,046 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1504): Waiting on 13c92efbf45fed23cebac57cf655e861, 1588230740, 8c7505042d76fb8c47639ac419e6a996, 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:04,046 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:56:04,047 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:56:04,047 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:56:04,047 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:56:04,047 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:56:04,047 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-19 05:56:04,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/cf1/f427cf18a9e642ea912f58012787edda 2023-07-19 05:56:04,065 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/info/93df9c78962948d6bb88838eddf93cf5 2023-07-19 05:56:04,083 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-19 05:56:04,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/norep/240a63796b644635b4c5a1e8d57b5bfd 2023-07-19 05:56:04,086 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/rep_barrier/262755a0cf6a4769ac7048541d602f79 2023-07-19 05:56:04,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/cf1/f427cf18a9e642ea912f58012787edda as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1/f427cf18a9e642ea912f58012787edda 2023-07-19 05:56:04,093 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-19 05:56:04,093 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/info/93df9c78962948d6bb88838eddf93cf5 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info/93df9c78962948d6bb88838eddf93cf5 2023-07-19 05:56:04,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/cf1/f427cf18a9e642ea912f58012787edda, entries=100, sequenceid=213, filesize=8.4 K 2023-07-19 05:56:04,104 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/info/93df9c78962948d6bb88838eddf93cf5, entries=24, sequenceid=31, filesize=7.4 K 2023-07-19 05:56:04,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/.tmp/norep/240a63796b644635b4c5a1e8d57b5bfd as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep/240a63796b644635b4c5a1e8d57b5bfd 2023-07-19 05:56:04,105 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/.tmp/rep_barrier/262755a0cf6a4769ac7048541d602f79 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier/262755a0cf6a4769ac7048541d602f79 2023-07-19 05:56:04,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/norep/240a63796b644635b4c5a1e8d57b5bfd, entries=1, sequenceid=213, filesize=4.8 K 2023-07-19 05:56:04,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for 8c7505042d76fb8c47639ac419e6a996 in 70ms, sequenceid=213, compaction requested=false 2023-07-19 05:56:04,117 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/rep_barrier/262755a0cf6a4769ac7048541d602f79, entries=2, sequenceid=31, filesize=4.9 K 2023-07-19 05:56:04,117 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 70ms, sequenceid=31, compaction requested=false 2023-07-19 05:56:04,132 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:56:04,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t1_syncup/8c7505042d76fb8c47639ac419e6a996/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-19 05:56:04,136 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-19 05:56:04,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:04,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8c7505042d76fb8c47639ac419e6a996: 2023-07-19 05:56:04,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689746123715.8c7505042d76fb8c47639ac419e6a996. 2023-07-19 05:56:04,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 13c92efbf45fed23cebac57cf655e861, disabling compactions & flushes 2023-07-19 05:56:04,137 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-19 05:56:04,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:04,137 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:56:04,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:04,138 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:56:04,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. after waiting 0 ms 2023-07-19 05:56:04,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:04,138 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-19 05:56:04,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/hbase/namespace/13c92efbf45fed23cebac57cf655e861/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-19 05:56:04,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:04,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 13c92efbf45fed23cebac57cf655e861: 2023-07-19 05:56:04,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689746119506.13c92efbf45fed23cebac57cf655e861. 2023-07-19 05:56:04,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8f71d242d3ffe63977903bdc0b5b9fe2, disabling compactions & flushes 2023-07-19 05:56:04,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:04,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:04,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. after waiting 0 ms 2023-07-19 05:56:04,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:04,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8f71d242d3ffe63977903bdc0b5b9fe2 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-19 05:56:04,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/cf1/ae1ac4fe131f483d88f864bf1d750080 2023-07-19 05:56:04,198 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-19 05:56:04,198 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-19 05:56:04,198 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664, lastWalPosition=728, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:56:04,198 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-19 05:56:04,247 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1504): Waiting on 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:04,303 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-19 05:56:04,384 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-19 05:56:04,408 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-19 05:56:04,447 DEBUG [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1504): Waiting on 8f71d242d3ffe63977903bdc0b5b9fe2 2023-07-19 05:56:04,512 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-19 05:56:04,527 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-19 05:56:04,527 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-19 05:56:04,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/norep/1aa690e740c14539bf4f3cda59ba5322 2023-07-19 05:56:04,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/cf1/ae1ac4fe131f483d88f864bf1d750080 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1/ae1ac4fe131f483d88f864bf1d750080 2023-07-19 05:56:04,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/cf1/ae1ac4fe131f483d88f864bf1d750080, entries=200, sequenceid=413, filesize=12.5 K 2023-07-19 05:56:04,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/.tmp/norep/1aa690e740c14539bf4f3cda59ba5322 as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep/1aa690e740c14539bf4f3cda59ba5322 2023-07-19 05:56:04,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/norep/1aa690e740c14539bf4f3cda59ba5322, entries=1, sequenceid=413, filesize=4.8 K 2023-07-19 05:56:04,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 8f71d242d3ffe63977903bdc0b5b9fe2 in 448ms, sequenceid=413, compaction requested=false 2023-07-19 05:56:04,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/data/default/t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-19 05:56:04,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:04,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8f71d242d3ffe63977903bdc0b5b9fe2: 2023-07-19 05:56:04,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689746124351.8f71d242d3ffe63977903bdc0b5b9fe2. 2023-07-19 05:56:04,616 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-19 05:56:04,647 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42035,1689746157262; all regions closed. 2023-07-19 05:56:04,651 DEBUG [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:56:04,651 INFO [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42035%2C1689746157262.meta:.meta(num 1689746157772) 2023-07-19 05:56:04,655 DEBUG [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:56:04,655 INFO [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1:(num 1689746162162) 2023-07-19 05:56:04,661 DEBUG [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs 2023-07-19 05:56:04,661 INFO [RS:0;jenkins-hbase4:42035] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0:(num 1689746157664) 2023-07-19 05:56:04,661 DEBUG [RS:0;jenkins-hbase4:42035] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:04,661 INFO [RS:0;jenkins-hbase4:42035] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:56:04,661 INFO [RS:0;jenkins-hbase4:42035] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-19 05:56:04,661 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:56:04,662 INFO [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-19 05:56:04,717 INFO [RS:0;jenkins-hbase4:42035.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/WALs/jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 was moved to hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 2023-07-19 05:56:04,764 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00030, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:04,764 DEBUG [RS:0;jenkins-hbase4:42035] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bb6bc80 to 127.0.0.1:65243 2023-07-19 05:56:04,764 DEBUG [RS:0;jenkins-hbase4:42035.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00030, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:04,764 DEBUG [RS:0;jenkins-hbase4:42035] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:04,784 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-19 05:56:04,866 INFO [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:42035.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1 terminated 2023-07-19 05:56:04,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-07-19 05:56:04,868 INFO [RS:0;jenkins-hbase4:42035] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1 terminated 2023-07-19 05:56:04,869 INFO [RS:0;jenkins-hbase4:42035] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42035 2023-07-19 05:56:04,874 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:04,874 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:56:04,874 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-19 05:56:04,875 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:56:04,875 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42035,1689746157262; numProcessing=1 2023-07-19 05:56:04,876 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42035,1689746157262 already deleted, retry=false 2023-07-19 05:56:04,876 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42035,1689746157262 expired; onlineServers=0 2023-07-19 05:56:04,876 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38455,1689746157233' ***** 2023-07-19 05:56:04,876 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-19 05:56:04,877 DEBUG [M:0;jenkins-hbase4:38455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@156b0cc1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:56:04,877 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38455,1689746157233 2023-07-19 05:56:04,877 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38455,1689746157233; all regions closed. 2023-07-19 05:56:04,877 DEBUG [M:0;jenkins-hbase4:38455] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:04,877 DEBUG [M:0;jenkins-hbase4:38455] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-19 05:56:04,877 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-19 05:56:04,877 DEBUG [M:0;jenkins-hbase4:38455] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-19 05:56:04,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746157449] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746157449,5,FailOnTimeoutGroup] 2023-07-19 05:56:04,878 INFO [M:0;jenkins-hbase4:38455] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-19 05:56:04,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746157449] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746157449,5,FailOnTimeoutGroup] 2023-07-19 05:56:04,879 INFO [M:0;jenkins-hbase4:38455] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-19 05:56:04,879 INFO [M:0;jenkins-hbase4:38455] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-19 05:56:04,879 DEBUG [M:0;jenkins-hbase4:38455] master.HMaster(1512): Stopping service threads 2023-07-19 05:56:04,879 INFO [M:0;jenkins-hbase4:38455] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-19 05:56:04,880 ERROR [M:0;jenkins-hbase4:38455] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-19 05:56:04,880 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-19 05:56:04,880 INFO [M:0;jenkins-hbase4:38455] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-19 05:56:04,880 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-19 05:56:04,880 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-19 05:56:04,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-19 05:56:04,881 DEBUG [M:0;jenkins-hbase4:38455] zookeeper.ZKUtil(398): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-19 05:56:04,881 WARN [M:0;jenkins-hbase4:38455] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-19 05:56:04,881 INFO [M:0;jenkins-hbase4:38455] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-19 05:56:04,881 INFO [M:0;jenkins-hbase4:38455] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-19 05:56:04,883 DEBUG [M:0;jenkins-hbase4:38455] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:56:04,883 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:04,883 DEBUG [M:0;jenkins-hbase4:38455] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:04,883 DEBUG [M:0;jenkins-hbase4:38455] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:56:04,883 DEBUG [M:0;jenkins-hbase4:38455] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:04,884 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.96 KB 2023-07-19 05:56:04,896 INFO [M:0;jenkins-hbase4:38455] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b824c6eab14c4ef792e4010d47d7dbeb 2023-07-19 05:56:04,901 DEBUG [M:0;jenkins-hbase4:38455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b824c6eab14c4ef792e4010d47d7dbeb as hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b824c6eab14c4ef792e4010d47d7dbeb 2023-07-19 05:56:04,906 INFO [M:0;jenkins-hbase4:38455] regionserver.HStore(1080): Added hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b824c6eab14c4ef792e4010d47d7dbeb, entries=10, sequenceid=218, filesize=5.8 K 2023-07-19 05:56:04,907 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30420, heapSize ~36.95 KB/37832, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=218, compaction requested=false 2023-07-19 05:56:04,909 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:04,909 DEBUG [M:0;jenkins-hbase4:38455] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:56:04,911 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/MasterData/WALs/jenkins-hbase4.apache.org,38455,1689746157233/jenkins-hbase4.apache.org%2C38455%2C1689746157233.jenkins-hbase4.apache.org%2C38455%2C1689746157233.regiongroup-0.1689746157347 not finished, retry = 0 2023-07-19 05:56:04,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:04,975 INFO [RS:0;jenkins-hbase4:42035] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42035,1689746157262; zookeeper connection closed. 2023-07-19 05:56:04,975 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:42035-0x1017c2601c0002c, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:04,976 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@199dbbba] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@199dbbba 2023-07-19 05:56:04,977 INFO [Listener at localhost/45003] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-19 05:56:05,012 INFO [M:0;jenkins-hbase4:38455] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-19 05:56:05,012 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:56:05,013 INFO [M:0;jenkins-hbase4:38455] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38455 2023-07-19 05:56:05,016 DEBUG [M:0;jenkins-hbase4:38455] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,38455,1689746157233 already deleted, retry=false 2023-07-19 05:56:05,117 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:05,117 INFO [M:0;jenkins-hbase4:38455] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38455,1689746157233; zookeeper connection closed. 2023-07-19 05:56:05,118 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:38455-0x1017c2601c0002b, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:05,121 INFO [Listener at localhost/45003] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:56:05,121 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:56:05,121 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:56:05,121 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:56:05,121 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:56:05,122 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:56:05,122 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:56:05,122 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41737 2023-07-19 05:56:05,123 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:56:05,124 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:56:05,124 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41737 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:56:05,127 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:417370x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:56:05,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41737-0x1017c2601c00033 connected 2023-07-19 05:56:05,129 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:56:05,130 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:56:05,130 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:56:05,131 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41737 2023-07-19 05:56:05,131 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41737 2023-07-19 05:56:05,131 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41737 2023-07-19 05:56:05,131 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41737 2023-07-19 05:56:05,131 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41737 2023-07-19 05:56:05,132 INFO [Listener at localhost/45003] master.HMaster(444): hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881, hbase.cluster.distributed=false 2023-07-19 05:56:05,132 DEBUG [pool-668-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-19 05:56:05,143 INFO [Listener at localhost/45003] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-19 05:56:05,144 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:56:05,144 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-19 05:56:05,144 INFO [Listener at localhost/45003] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-19 05:56:05,144 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-19 05:56:05,144 INFO [Listener at localhost/45003] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-19 05:56:05,144 INFO [Listener at localhost/45003] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-19 05:56:05,145 INFO [Listener at localhost/45003] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44553 2023-07-19 05:56:05,145 INFO [Listener at localhost/45003] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-19 05:56:05,146 DEBUG [Listener at localhost/45003] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-19 05:56:05,146 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:56:05,147 INFO [Listener at localhost/45003] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:56:05,148 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44553 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:56:05,151 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:445530x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:56:05,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44553-0x1017c2601c00034 connected 2023-07-19 05:56:05,154 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:56:05,154 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:56:05,155 DEBUG [Listener at localhost/45003] zookeeper.ZKUtil(164): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-19 05:56:05,155 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44553 2023-07-19 05:56:05,155 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44553 2023-07-19 05:56:05,155 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44553 2023-07-19 05:56:05,156 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44553 2023-07-19 05:56:05,156 DEBUG [Listener at localhost/45003] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44553 2023-07-19 05:56:05,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,158 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-19 05:56:05,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,162 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:56:05,162 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-19 05:56:05,162 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:56:05,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-19 05:56:05,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,41737,1689746165120 from backup master directory 2023-07-19 05:56:05,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-19 05:56:05,166 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,166 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-19 05:56:05,166 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:56:05,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-19 05:56:05,202 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6769c862 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:05,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4cec5ef4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:05,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-19 05:56:05,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-19 05:56:05,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:56:05,206 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:56:05,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485 to hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485-dead as it is dead 2023-07-19 05:56:05,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485-dead/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 2023-07-19 05:56:05,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485-dead/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 after 0ms 2023-07-19 05:56:05,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485-dead/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 to hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 2023-07-19 05:56:05,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,34645,1689746130485-dead 2023-07-19 05:56:05,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:56:05,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41737%2C1689746165120.jenkins-hbase4.apache.org%2C41737%2C1689746165120.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,41737,1689746165120, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/oldWALs, maxLogs=10 2023-07-19 05:56:05,227 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:56:05,227 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:56:05,227 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:56:05,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/WALs/jenkins-hbase4.apache.org,41737,1689746165120/jenkins-hbase4.apache.org%2C41737%2C1689746165120.jenkins-hbase4.apache.org%2C41737%2C1689746165120.regiongroup-0.1689746165215 2023-07-19 05:56:05,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK]] 2023-07-19 05:56:05,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:05,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:05,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:56:05,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:56:05,232 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:56:05,232 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-19 05:56:05,233 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-19 05:56:05,238 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 23a3c3ef4a3e4624823144fba98d1ab1 2023-07-19 05:56:05,238 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/23a3c3ef4a3e4624823144fba98d1ab1 2023-07-19 05:56:05,241 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d3977543fb84f4c9cc62c6480faff0b 2023-07-19 05:56:05,242 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:05,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-19 05:56:05,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 2023-07-19 05:56:05,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 140, firstSequenceIdInLog=120, maxSequenceIdInLog=241, path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 2023-07-19 05:56:05,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34645%2C1689746130485.jenkins-hbase4.apache.org%2C34645%2C1689746130485.regiongroup-0.1689746130595 2023-07-19 05:56:05,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-19 05:56:05,250 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/241.seqid, newMaxSeqId=241, maxSeqId=118 2023-07-19 05:56:05,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=242; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60561906, jitterRate=-0.09755727648735046}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:56:05,251 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:56:05,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-19 05:56:05,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-19 05:56:05,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-19 05:56:05,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-19 05:56:05,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-19 05:56:05,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-19 05:56:05,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-19 05:56:05,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-19 05:56:05,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-19 05:56:05,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-19 05:56:05,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-19 05:56:05,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-19 05:56:05,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-19 05:56:05,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-19 05:56:05,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38293,1689746130512, table=hbase:meta, region=1588230740 2023-07-19 05:56:05,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-19 05:56:05,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,38293,1689746130512 already deleted, retry=false 2023-07-19 05:56:05,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,38293,1689746130512 on jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,271 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,38293,1689746130512, splitWal=true, meta=true 2023-07-19 05:56:05,271 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,38293,1689746130512 (carryingMeta=true) jenkins-hbase4.apache.org,38293,1689746130512/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1949a95a[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-19 05:56:05,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-19 05:56:05,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-19 05:56:05,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-19 05:56:05,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-19 05:56:05,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-19 05:56:05,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-19 05:56:05,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-19 05:56:05,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-19 05:56:05,287 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:56:05,287 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:56:05,287 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-19 05:56:05,287 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41737,1689746165120, sessionid=0x1017c2601c00033, setting cluster-up flag (Was=false) 2023-07-19 05:56:05,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-19 05:56:05,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-19 05:56:05,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:05,298 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/.hbase-snapshot/.tmp 2023-07-19 05:56:05,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:56:05,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689746195308 2023-07-19 05:56:05,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-19 05:56:05,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-19 05:56:05,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-19 05:56:05,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-19 05:56:05,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-19 05:56:05,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-19 05:56:05,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-19 05:56:05,311 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38293,1689746130512; numProcessing=1 2023-07-19 05:56:05,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-19 05:56:05,311 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38293,1689746130512, splitWal=true, meta=true 2023-07-19 05:56:05,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-19 05:56:05,312 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38293,1689746130512, splitWal=true, meta=true, isMeta: true 2023-07-19 05:56:05,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-19 05:56:05,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-19 05:56:05,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746165315,5,FailOnTimeoutGroup] 2023-07-19 05:56:05,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746165315,5,FailOnTimeoutGroup] 2023-07-19 05:56:05,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-19 05:56:05,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689746165317, completionTime=-1 2023-07-19 05:56:05,317 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-19 05:56:05,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-19 05:56:05,317 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512-splitting 2023-07-19 05:56:05,318 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512-splitting dir is empty, no logs to split. 2023-07-19 05:56:05,318 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38293,1689746130512 WAL count=0, meta=true 2023-07-19 05:56:05,320 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512-splitting dir is empty, no logs to split. 2023-07-19 05:56:05,320 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38293,1689746130512 WAL count=0, meta=true 2023-07-19 05:56:05,320 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38293,1689746130512 WAL splitting is done? wals=0, meta=true 2023-07-19 05:56:05,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-19 05:56:05,322 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-19 05:56:05,323 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:05,357 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(951): ClusterId : 665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:56:05,357 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-19 05:56:05,359 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-19 05:56:05,359 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-19 05:56:05,361 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-19 05:56:05,363 DEBUG [RS:0;jenkins-hbase4:44553] zookeeper.ReadOnlyZKClient(139): Connect 0x79db49d5 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:05,369 DEBUG [RS:0;jenkins-hbase4:44553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@771cb89e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:05,369 DEBUG [RS:0;jenkins-hbase4:44553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@732322a5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:56:05,377 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44553 2023-07-19 05:56:05,378 INFO [RS:0;jenkins-hbase4:44553] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-19 05:56:05,378 INFO [RS:0;jenkins-hbase4:44553] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-19 05:56:05,378 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1022): About to register with Master. 2023-07-19 05:56:05,378 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41737,1689746165120 with isa=jenkins-hbase4.apache.org/172.31.14.131:44553, startcode=1689746165143 2023-07-19 05:56:05,378 DEBUG [RS:0;jenkins-hbase4:44553] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-19 05:56:05,380 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39561, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-19 05:56:05,381 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41737] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,382 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881 2023-07-19 05:56:05,382 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33839 2023-07-19 05:56:05,382 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-19 05:56:05,383 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c0002a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:05,383 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00027, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:05,383 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:05,383 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:05,383 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:05,383 DEBUG [RS:0;jenkins-hbase4:44553] zookeeper.ZKUtil(162): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,383 WARN [RS:0;jenkins-hbase4:44553] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-19 05:56:05,383 INFO [RS:0;jenkins-hbase4:44553] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:56:05,384 INFO [RS:0;jenkins-hbase4:44553] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:56:05,384 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,385 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44553,1689746165143] 2023-07-19 05:56:05,388 DEBUG [RS:0;jenkins-hbase4:44553] zookeeper.ZKUtil(162): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,388 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:56:05,389 INFO [RS:0;jenkins-hbase4:44553] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-19 05:56:05,390 INFO [RS:0;jenkins-hbase4:44553] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-19 05:56:05,390 INFO [RS:0;jenkins-hbase4:44553] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-19 05:56:05,390 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,390 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-19 05:56:05,391 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,391 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,392 DEBUG [RS:0;jenkins-hbase4:44553] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-19 05:56:05,393 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,393 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,393 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,406 INFO [RS:0;jenkins-hbase4:44553] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-19 05:56:05,406 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44553,1689746165143-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:05,416 INFO [RS:0;jenkins-hbase4:44553] regionserver.Replication(203): jenkins-hbase4.apache.org,44553,1689746165143 started 2023-07-19 05:56:05,416 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44553,1689746165143, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44553, sessionid=0x1017c2601c00034 2023-07-19 05:56:05,416 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-19 05:56:05,416 DEBUG [RS:0;jenkins-hbase4:44553] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,416 DEBUG [RS:0;jenkins-hbase4:44553] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44553,1689746165143' 2023-07-19 05:56:05,416 DEBUG [RS:0;jenkins-hbase4:44553] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-19 05:56:05,416 DEBUG [RS:0;jenkins-hbase4:44553] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44553,1689746165143' 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-19 05:56:05,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-19 05:56:05,417 DEBUG [RS:0;jenkins-hbase4:44553] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-19 05:56:05,417 INFO [RS:0;jenkins-hbase4:44553] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-19 05:56:05,417 INFO [RS:0;jenkins-hbase4:44553] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-19 05:56:05,473 DEBUG [jenkins-hbase4:41737] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-19 05:56:05,475 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44553,1689746165143, state=OPENING 2023-07-19 05:56:05,477 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-19 05:56:05,477 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-19 05:56:05,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44553,1689746165143}] 2023-07-19 05:56:05,517 INFO [RS:0;jenkins-hbase4:44553] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:56:05,519 INFO [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44553%2C1689746165143.jenkins-hbase4.apache.org%2C44553%2C1689746165143.regiongroup-0, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:56:05,542 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:56:05,542 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:56:05,542 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:56:05,544 INFO [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143/jenkins-hbase4.apache.org%2C44553%2C1689746165143.jenkins-hbase4.apache.org%2C44553%2C1689746165143.regiongroup-0.1689746165520 2023-07-19 05:56:05,545 DEBUG [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK]] 2023-07-19 05:56:05,633 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:05,634 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:56:05,635 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43906, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:56:05,638 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-19 05:56:05,638 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:56:05,638 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:56:05,638 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:56:05,640 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44553%2C1689746165143.meta, suffix=.meta, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:56:05,652 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:56:05,652 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK] 2023-07-19 05:56:05,652 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:56:05,654 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143/jenkins-hbase4.apache.org%2C44553%2C1689746165143.meta.1689746165640.meta 2023-07-19 05:56:05,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:37059,DS-233e2b8f-03e2-4aa3-9f7e-6baac92a0f7e,DISK], DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK]] 2023-07-19 05:56:05,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:05,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-19 05:56:05,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-19 05:56:05,654 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-19 05:56:05,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-19 05:56:05,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:05,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-19 05:56:05,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-19 05:56:05,656 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-19 05:56:05,657 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:56:05,657 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info 2023-07-19 05:56:05,657 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-19 05:56:05,665 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/adcf10598c134d8f9bf188f7c94885aa 2023-07-19 05:56:05,670 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/fe9f1349700b4d27bc86accbfe86fb7c 2023-07-19 05:56:05,670 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:05,670 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-19 05:56:05,671 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:56:05,671 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/rep_barrier 2023-07-19 05:56:05,671 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-19 05:56:05,672 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:05,672 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-19 05:56:05,673 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:56:05,673 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table 2023-07-19 05:56:05,673 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-19 05:56:05,677 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/table/5ac4cf9a6573436f8499f24426169c83 2023-07-19 05:56:05,677 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:05,678 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:56:05,679 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740 2023-07-19 05:56:05,681 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-19 05:56:05,682 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-19 05:56:05,683 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71994900, jitterRate=0.07280761003494263}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-19 05:56:05,683 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-19 05:56:05,684 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689746165633 2023-07-19 05:56:05,688 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-19 05:56:05,688 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-19 05:56:05,691 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44553,1689746165143, state=OPEN 2023-07-19 05:56:05,692 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-19 05:56:05,692 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-19 05:56:05,694 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-19 05:56:05,694 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44553,1689746165143 in 216 msec 2023-07-19 05:56:05,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-19 05:56:05,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 374 msec 2023-07-19 05:56:06,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1602ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-19 05:56:07,666 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:56:08,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-19 05:56:09,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-19 05:56:09,824 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-19 05:56:09,827 DEBUG [hconnection-0x64f21a96-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:09,829 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43922, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:56:09,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ecfca84342d05307f433bda44290dd5a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38293,1689746130512, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512, openSeqNum=10 2023-07-19 05:56:09,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=2054d9ec275367d29389fbedb9c09867, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38293,1689746130512, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512, openSeqNum=11 2023-07-19 05:56:09,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38293,1689746130512, regionLocation=jenkins-hbase4.apache.org,38293,1689746130512, openSeqNum=13 2023-07-19 05:56:09,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-19 05:56:09,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689746229835 2023-07-19 05:56:09,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689746289835 2023-07-19 05:56:09,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-19 05:56:09,851 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,38293,1689746130512 had 4 regions 2023-07-19 05:56:09,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41737,1689746165120-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:09,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41737,1689746165120-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:09,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41737,1689746165120-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:09,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41737, period=300000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:09,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-19 05:56:09,852 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. is NOT online; state={ecfca84342d05307f433bda44290dd5a state=OPEN, ts=1689746169835, server=jenkins-hbase4.apache.org,38293,1689746130512}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-19 05:56:09,852 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38293,1689746130512, splitWal=true, meta=true, isMeta: false 2023-07-19 05:56:09,854 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,38293,1689746130512/hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., unknown_server=jenkins-hbase4.apache.org,38293,1689746130512/t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867., unknown_server=jenkins-hbase4.apache.org,38293,1689746130512/t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:09,855 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512-splitting dir is empty, no logs to split. 2023-07-19 05:56:09,855 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,38293,1689746130512 WAL count=0, meta=false 2023-07-19 05:56:09,857 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,38293,1689746130512-splitting dir is empty, no logs to split. 2023-07-19 05:56:09,857 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,38293,1689746130512 WAL count=0, meta=false 2023-07-19 05:56:09,857 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38293,1689746130512 WAL splitting is done? wals=0, meta=false 2023-07-19 05:56:09,860 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN}] 2023-07-19 05:56:09,861 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN 2023-07-19 05:56:09,861 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN 2023-07-19 05:56:09,861 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN 2023-07-19 05:56:09,862 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:09,862 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:09,862 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-19 05:56:10,012 DEBUG [jenkins-hbase4:41737] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-19 05:56:10,016 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=ecfca84342d05307f433bda44290dd5a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:10,016 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:10,016 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2054d9ec275367d29389fbedb9c09867, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:10,016 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746170016"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746170016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746170016"}]},"ts":"1689746170016"} 2023-07-19 05:56:10,016 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746170016"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746170016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746170016"}]},"ts":"1689746170016"} 2023-07-19 05:56:10,016 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746170016"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689746170016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689746170016"}]},"ts":"1689746170016"} 2023-07-19 05:56:10,018 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure ecfca84342d05307f433bda44290dd5a, server=jenkins-hbase4.apache.org,44553,1689746165143}] 2023-07-19 05:56:10,019 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 168d6c8e660cdb03a471b9cc5e7a3e2b, server=jenkins-hbase4.apache.org,44553,1689746165143}] 2023-07-19 05:56:10,023 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=31, state=RUNNABLE; OpenRegionProcedure 2054d9ec275367d29389fbedb9c09867, server=jenkins-hbase4.apache.org,44553,1689746165143}] 2023-07-19 05:56:10,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:10,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-19 05:56:10,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44553%2C1689746165143.jenkins-hbase4.apache.org%2C44553%2C1689746165143.regiongroup-1, suffix=, logDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143, archiveDir=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs, maxLogs=10 2023-07-19 05:56:10,181 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:10,181 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ecfca84342d05307f433bda44290dd5a, NAME => 'hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:10,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:10,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,189 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,190 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info 2023-07-19 05:56:10,191 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info 2023-07-19 05:56:10,191 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ecfca84342d05307f433bda44290dd5a columnFamilyName info 2023-07-19 05:56:10,197 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK] 2023-07-19 05:56:10,197 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK] 2023-07-19 05:56:10,204 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK] 2023-07-19 05:56:10,208 DEBUG [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/info/12ba01b16a514d88854eada26bfe99ee 2023-07-19 05:56:10,219 INFO [StoreOpener-ecfca84342d05307f433bda44290dd5a-1] regionserver.HStore(310): Store=ecfca84342d05307f433bda44290dd5a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:10,220 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,225 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:10,226 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ecfca84342d05307f433bda44290dd5a; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59744404, jitterRate=-0.10973900556564331}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-19 05:56:10,226 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:56:10,227 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., pid=33, masterSystemTime=1689746170173 2023-07-19 05:56:10,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/WALs/jenkins-hbase4.apache.org,44553,1689746165143/jenkins-hbase4.apache.org%2C44553%2C1689746165143.jenkins-hbase4.apache.org%2C44553%2C1689746165143.regiongroup-1.1689746170180 2023-07-19 05:56:10,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43005,DS-9a06766e-3818-433b-ad90-54a618b3c648,DISK], DatanodeInfoWithStorage[127.0.0.1:35049,DS-35686ffa-ada0-46f2-b5b2-f8da3b859896,DISK], DatanodeInfoWithStorage[127.0.0.1:33225,DS-1adb813e-2575-47a2-84ac-2cef5ee08feb,DISK]] 2023-07-19 05:56:10,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 168d6c8e660cdb03a471b9cc5e7a3e2b, NAME => 't2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:10,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:10,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,237 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:10,237 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:10,238 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=ecfca84342d05307f433bda44290dd5a, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:10,238 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689746170238"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746170238"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746170238"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746170238"}]},"ts":"1689746170238"} 2023-07-19 05:56:10,243 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-07-19 05:56:10,243 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure ecfca84342d05307f433bda44290dd5a, server=jenkins-hbase4.apache.org,44553,1689746165143 in 222 msec 2023-07-19 05:56:10,247 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,247 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ecfca84342d05307f433bda44290dd5a, ASSIGN in 385 msec 2023-07-19 05:56:10,249 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1 2023-07-19 05:56:10,249 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1 2023-07-19 05:56:10,249 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 168d6c8e660cdb03a471b9cc5e7a3e2b columnFamilyName cf1 2023-07-19 05:56:10,282 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ea0ffeb6d72499d849182a16f9441a7 2023-07-19 05:56:10,282 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/2ea0ffeb6d72499d849182a16f9441a7 2023-07-19 05:56:10,290 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/f3d6786e81f546b7b78459c70b2928c5 2023-07-19 05:56:10,290 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(310): Store=168d6c8e660cdb03a471b9cc5e7a3e2b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:10,290 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,291 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/norep 2023-07-19 05:56:10,292 DEBUG [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/norep 2023-07-19 05:56:10,292 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 168d6c8e660cdb03a471b9cc5e7a3e2b columnFamilyName norep 2023-07-19 05:56:10,293 INFO [StoreOpener-168d6c8e660cdb03a471b9cc5e7a3e2b-1] regionserver.HStore(310): Store=168d6c8e660cdb03a471b9cc5e7a3e2b/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:10,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:56:10,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:10,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 168d6c8e660cdb03a471b9cc5e7a3e2b; next sequenceid=20; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71376542, jitterRate=0.06359335780143738}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:56:10,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:56:10,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b., pid=34, masterSystemTime=1689746170173 2023-07-19 05:56:10,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:10,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:10,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:10,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2054d9ec275367d29389fbedb9c09867, NAME => 't1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.', STARTKEY => '', ENDKEY => ''} 2023-07-19 05:56:10,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-19 05:56:10,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,306 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=168d6c8e660cdb03a471b9cc5e7a3e2b, regionState=OPEN, openSeqNum=20, regionLocation=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:10,306 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746170306"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746170306"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746170306"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746170306"}]},"ts":"1689746170306"} 2023-07-19 05:56:10,310 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-19 05:56:10,310 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 168d6c8e660cdb03a471b9cc5e7a3e2b, server=jenkins-hbase4.apache.org,44553,1689746165143 in 289 msec 2023-07-19 05:56:10,312 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=168d6c8e660cdb03a471b9cc5e7a3e2b, ASSIGN in 453 msec 2023-07-19 05:56:10,319 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,321 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1 2023-07-19 05:56:10,321 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1 2023-07-19 05:56:10,322 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2054d9ec275367d29389fbedb9c09867 columnFamilyName cf1 2023-07-19 05:56:10,361 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 281b662bf8104a34817e5c08d82ce4a0 2023-07-19 05:56:10,361 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/281b662bf8104a34817e5c08d82ce4a0 2023-07-19 05:56:10,368 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(539): loaded hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/56ea8d42fc7f467a8b4c5beb4118c066 2023-07-19 05:56:10,368 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(310): Store=2054d9ec275367d29389fbedb9c09867/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:10,369 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,370 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/norep 2023-07-19 05:56:10,370 DEBUG [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/norep 2023-07-19 05:56:10,371 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2054d9ec275367d29389fbedb9c09867 columnFamilyName norep 2023-07-19 05:56:10,371 INFO [StoreOpener-2054d9ec275367d29389fbedb9c09867-1] regionserver.HStore(310): Store=2054d9ec275367d29389fbedb9c09867/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-19 05:56:10,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-19 05:56:10,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:10,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2054d9ec275367d29389fbedb9c09867; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60847145, jitterRate=-0.09330688416957855}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-19 05:56:10,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:56:10,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867., pid=35, masterSystemTime=1689746170173 2023-07-19 05:56:10,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:10,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:10,383 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2054d9ec275367d29389fbedb9c09867, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:10,384 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689746170383"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689746170383"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689746170383"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689746170383"}]},"ts":"1689746170383"} 2023-07-19 05:56:10,387 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=31 2023-07-19 05:56:10,387 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=31, state=SUCCESS; OpenRegionProcedure 2054d9ec275367d29389fbedb9c09867, server=jenkins-hbase4.apache.org,44553,1689746165143 in 365 msec 2023-07-19 05:56:10,389 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=27 2023-07-19 05:56:10,389 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,38293,1689746130512 after splitting done 2023-07-19 05:56:10,389 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=2054d9ec275367d29389fbedb9c09867, ASSIGN in 529 msec 2023-07-19 05:56:10,390 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,38293,1689746130512 from processing; numProcessing=0 2023-07-19 05:56:10,392 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,38293,1689746130512, splitWal=true, meta=true in 5.1240 sec 2023-07-19 05:56:10,416 INFO [jenkins-hbase4:44553Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:56:10,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-19 05:56:10,866 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-19 05:56:10,874 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-19 05:56:10,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.707sec 2023-07-19 05:56:10,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-19 05:56:10,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-19 05:56:10,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-19 05:56:10,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41737,1689746165120-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-19 05:56:10,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41737,1689746165120-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-19 05:56:10,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-19 05:56:10,961 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x10db91c1 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:10,966 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@199b1925, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:10,967 DEBUG [hconnection-0x3034855-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:10,969 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45406, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:56:10,971 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-19 05:56:10,971 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10db91c1 to 127.0.0.1:65243 2023-07-19 05:56:10,971 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:10,972 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:41737 after: jenkins-hbase4.apache.org:41737 2023-07-19 05:56:10,973 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(139): Connect 0x6f448f63 to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:10,977 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@761c233e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:11,389 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-19 05:56:11,389 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-19 05:56:11,390 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-19 05:56:11,390 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-19 05:56:11,479 DEBUG [hconnection-0x658c7798-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:11,481 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45408, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-19 05:56:11,500 INFO [Listener at localhost/45003] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689746171500 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:56:11,502 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication16897461715000x0, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:56:11,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689746171500-0x1017c2601c00039 connected Start Replication Server start 2023-07-19 05:56:11,505 INFO [Listener at localhost/45003] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-19 05:56:11,505 INFO [Listener at localhost/45003] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-19 05:56:11,517 INFO [Listener at localhost/45003] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-19 05:56:11,518 DEBUG [Listener at localhost/45003] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-19 05:56:11,529 INFO [Listener at localhost/45003] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:56:11,530 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,42035,1689746157262] other RSs: [] 2023-07-19 05:56:11,549 DEBUG [Listener at localhost/45003.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x110dbf7a to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:11,553 DEBUG [Listener at localhost/45003.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@314106eb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:11,553 INFO [Listener at localhost/45003.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:56:11,557 DEBUG [Listener at localhost/45003.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:56:11,559 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c0003b connected 2023-07-19 05:56:11,559 INFO [Listener at localhost/45003.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:56:14,164 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42035,1689746157262/1's WALs to 1689746171505.syncuptool.replication.org,1234,1 2023-07-19 05:56:14,166 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162 with data PBUF\x08\xB7\x06 2023-07-19 05:56:14,166 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 with data PBUF\x08\xD8\x05 2023-07-19 05:56:14,171 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,42035,1689746157262/1's WALs to 1689746171505.syncuptool.replication.org,1234,1 2023-07-19 05:56:15,416 INFO [jenkins-hbase4:44553Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:56:15,684 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:56:16,173 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:56:16,173 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:56:16,184 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:56:16,185 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,42035,1689746157262, ReplicationSource: 1, currentBandwidth=0 2023-07-19 05:56:16,185 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,204 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] zookeeper.ReadOnlyZKClient(139): Connect 0x7c65aeff to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:16,208 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@30d5f282, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:16,208 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:65243 2023-07-19 05:56:16,211 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-19 05:56:16,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017c2601c0003d connected 2023-07-19 05:56:16,212 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,42035,1689746157262 (queues=2) is replicating from cluster=fa474148-f7f6-4a32-a2c9-9f3f924e734e to cluster=665b6c74-789a-4eb9-8c1c-1e4193b2244d 2023-07-19 05:56:16,212 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0 2023-07-19 05:56:16,215 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42035,1689746157262, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:56:16,216 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1 2023-07-19 05:56:16,216 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0 2023-07-19 05:56:16,217 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42035,1689746157262, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-19 05:56:16,218 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1 2023-07-19 05:56:16,221 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 2023-07-19 05:56:16,221 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-19 05:56:16,222 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42035,1689746157262 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:56:16,222 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,227 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162 2023-07-19 05:56:16,227 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-19 05:56:16,227 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-19 05:56:16,229 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42035,1689746157262 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/8c7505042d76fb8c47639ac419e6a996/112=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/113=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/114=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/115=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/116=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/117=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/118=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/119=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/120=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/121=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/122=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/123=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/124=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/125=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/126=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/127=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/128=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/129=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/130=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/131=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/132=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/133=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/134=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/135=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/136=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/137=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/138=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/139=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/140=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/141=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/142=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/143=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/144=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/145=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/146=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/147=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/148=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/149=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/150=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/151=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/152=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/153=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/154=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/155=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/156=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/157=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/158=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/159=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/160=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/161=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/162=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/163=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/164=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/165=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/166=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/167=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/168=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/169=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/170=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/171=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/172=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/173=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/174=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/175=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/176=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/177=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/178=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/179=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/180=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/181=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/182=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/183=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/184=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/185=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/186=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/187=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/188=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/189=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/190=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/191=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/192=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/193=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/194=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/195=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/196=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/197=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/198=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/199=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/200=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/201=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/202=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/203=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/204=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/205=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/206=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/207=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/208=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/209=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/210=[#edits: 1 = ],208}, {t1_syncup/8c7505042d76fb8c47639ac419e6a996/211=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/212=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/213=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/214=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/215=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/216=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/217=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/218=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/219=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/220=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/221=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/222=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/223=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/224=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/225=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/226=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/227=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/228=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/229=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/230=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/231=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/232=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/233=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/234=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/235=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/236=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/237=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/238=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/239=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/240=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/241=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/242=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/243=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/244=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/245=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/246=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/247=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/248=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/249=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/250=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/251=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/252=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/253=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/254=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/255=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/256=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/257=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/258=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/259=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/260=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/261=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/262=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/263=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/264=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/265=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/266=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/267=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/268=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/269=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/270=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/271=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/272=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/273=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/274=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/275=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/276=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/277=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/278=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/279=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/280=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/281=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/282=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/283=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/284=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/285=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/286=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/287=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/288=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/289=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/290=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/291=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/292=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/293=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/294=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/295=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/296=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/297=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/298=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/299=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/300=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/301=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/302=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/303=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/304=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/305=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/306=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/307=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/308=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/309=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/310=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/311=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/312=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/313=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/314=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/315=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/316=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/317=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/318=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/319=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/320=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/321=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/322=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/323=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/324=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/325=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/326=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/327=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/328=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/329=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/330=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/331=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/332=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/333=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/334=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/335=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/336=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/337=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/338=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/339=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/340=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/341=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/342=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/343=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/344=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/345=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/346=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/347=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/348=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/349=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/350=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/351=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/352=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/353=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/354=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/355=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/356=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/357=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/358=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/359=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/360=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/361=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/362=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/363=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/364=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/365=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/366=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/367=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/368=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/369=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/370=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/371=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/372=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/373=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/374=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/375=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/376=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/377=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/378=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/379=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/380=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/381=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/382=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/383=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/384=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/385=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/386=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/387=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/388=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/389=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/390=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/391=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/392=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/393=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/394=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/395=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/396=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/397=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/398=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/399=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/400=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/401=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/402=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/403=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/404=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/405=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/406=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/407=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/408=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/409=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/410=[#edits: 1 = ],208}, {t2_syncup/8f71d242d3ffe63977903bdc0b5b9fe2/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:46083/user/jenkins/test-data/3c736770-85b5-2879-0034-e0c71c654320/oldWALs/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162, lastWalPosition=34707, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-19 05:56:16,234 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-19 05:56:16,236 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45422, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-19 05:56:16,240 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44553] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-19 05:56:16,240 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44553] zookeeper.ReadOnlyZKClient(139): Connect 0x756a878c to 127.0.0.1:65243 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-19 05:56:16,244 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7bad7751, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-19 05:56:16,245 DEBUG [hconnection-0x7f92ecfc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-19 05:56:16,247 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45428, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-19 05:56:16,266 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44553] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-19 05:56:16,266 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42035,1689746157262] 2023-07-19 05:56:16,268 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1.1689746162162] 2023-07-19 05:56:16,270 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42035,1689746157262 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-19 05:56:16,270 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,322 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,323 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] zookeeper.ZKUtil(162): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689746171505.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 2023-07-19 05:56:16,326 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689746171505.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42035,1689746157262/jenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0.1689746157664 2023-07-19 05:56:16,326 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689746171505.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,326 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-0,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42035,1689746157262 with the following stats: Total replicated edits: 300, current progress: 2023-07-19 05:56:16,326 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689746171505.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,426 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42035,1689746157262 2023-07-19 05:56:16,427 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] zookeeper.ZKUtil(243): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689746171505.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42035,1689746157262 because node does not exist (not an error) 2023-07-19 05:56:16,427 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262.replicationSource.shipperjenkins-hbase4.apache.org%2C42035%2C1689746157262.jenkins-hbase4.apache.org%2C42035%2C1689746157262.regiongroup-1,1-jenkins-hbase4.apache.org,42035,1689746157262] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42035,1689746157262 with the following stats: Total replicated edits: 300, current progress: 2023-07-19 05:56:20,416 INFO [jenkins-hbase4:44553Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-19 05:56:20,416 INFO [jenkins-hbase4:44553Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-19 05:56:21,531 INFO [Listener at localhost/45003] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-19 05:56:21,634 DEBUG [Listener at localhost/45003.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c0003b, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:21,634 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x110dbf7a to 127.0.0.1:65243 2023-07-19 05:56:21,636 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:21,825 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-19 05:56:21,837 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:21,838 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689746171500-0x1017c2601c00039, quorum=127.0.0.1:65243, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:21,845 INFO [Listener at localhost/45003] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-19 05:56:21,846 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-19 05:56:21,846 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f448f63 to 127.0.0.1:65243 2023-07-19 05:56:21,846 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:21,846 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-19 05:56:21,846 DEBUG [Listener at localhost/45003] util.JVMClusterUtil(257): Found active master hash=1409907672, stopped=false 2023-07-19 05:56:21,846 INFO [Listener at localhost/45003] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:21,848 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:56:21,848 INFO [Listener at localhost/45003] procedure2.ProcedureExecutor(629): Stopping 2023-07-19 05:56:21,848 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:56:21,848 DEBUG [Listener at localhost/45003] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6769c862 to 127.0.0.1:65243 2023-07-19 05:56:21,848 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-19 05:56:21,851 DEBUG [Listener at localhost/45003] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:21,851 INFO [Listener at localhost/45003] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44553,1689746165143' ***** 2023-07-19 05:56:21,851 INFO [Listener at localhost/45003] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-19 05:56:21,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:56:21,852 INFO [RS:0;jenkins-hbase4:44553] regionserver.HeapMemoryManager(220): Stopping 2023-07-19 05:56:21,852 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-19 05:56:21,852 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-19 05:56:21,852 INFO [RS:0;jenkins-hbase4:44553] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-19 05:56:21,853 INFO [RS:0;jenkins-hbase4:44553] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-19 05:56:21,853 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(3305): Received CLOSE for 168d6c8e660cdb03a471b9cc5e7a3e2b 2023-07-19 05:56:21,855 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(3305): Received CLOSE for ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:21,855 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(3305): Received CLOSE for 2054d9ec275367d29389fbedb9c09867 2023-07-19 05:56:21,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 168d6c8e660cdb03a471b9cc5e7a3e2b, disabling compactions & flushes 2023-07-19 05:56:21,856 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:21,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:21,856 DEBUG [RS:0;jenkins-hbase4:44553] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x79db49d5 to 127.0.0.1:65243 2023-07-19 05:56:21,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:21,856 DEBUG [RS:0;jenkins-hbase4:44553] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:21,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. after waiting 0 ms 2023-07-19 05:56:21,856 INFO [RS:0;jenkins-hbase4:44553] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-19 05:56:21,857 INFO [RS:0;jenkins-hbase4:44553] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-19 05:56:21,857 INFO [RS:0;jenkins-hbase4:44553] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-19 05:56:21,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:21,857 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-19 05:56:21,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 168d6c8e660cdb03a471b9cc5e7a3e2b 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-19 05:56:21,857 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-19 05:56:21,857 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1478): Online Regions={168d6c8e660cdb03a471b9cc5e7a3e2b=t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b., ecfca84342d05307f433bda44290dd5a=hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a., 1588230740=hbase:meta,,1.1588230740, 2054d9ec275367d29389fbedb9c09867=t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867.} 2023-07-19 05:56:21,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-19 05:56:21,858 DEBUG [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1504): Waiting on 1588230740, 168d6c8e660cdb03a471b9cc5e7a3e2b, 2054d9ec275367d29389fbedb9c09867, ecfca84342d05307f433bda44290dd5a 2023-07-19 05:56:21,858 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-19 05:56:21,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-19 05:56:21,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-19 05:56:21,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-19 05:56:21,858 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-19 05:56:21,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=23 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/.tmp/cf1/697485a5c9ab41f989889d5ec7added5 2023-07-19 05:56:21,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/info/f4cf40189ecb4cf38fe30b5090c23a6a 2023-07-19 05:56:21,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/.tmp/cf1/697485a5c9ab41f989889d5ec7added5 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/697485a5c9ab41f989889d5ec7added5 2023-07-19 05:56:21,902 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/.tmp/info/f4cf40189ecb4cf38fe30b5090c23a6a as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/f4cf40189ecb4cf38fe30b5090c23a6a 2023-07-19 05:56:21,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/cf1/697485a5c9ab41f989889d5ec7added5, entries=200, sequenceid=23, filesize=12.2 K 2023-07-19 05:56:21,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 168d6c8e660cdb03a471b9cc5e7a3e2b in 50ms, sequenceid=23, compaction requested=true 2023-07-19 05:56:21,908 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/info/f4cf40189ecb4cf38fe30b5090c23a6a, entries=24, sequenceid=43, filesize=7.4 K 2023-07-19 05:56:21,909 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:56:21,917 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 59ms, sequenceid=43, compaction requested=true 2023-07-19 05:56:21,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t2_syncup/168d6c8e660cdb03a471b9cc5e7a3e2b/recovered.edits/26.seqid, newMaxSeqId=26, maxSeqId=19 2023-07-19 05:56:21,936 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-19 05:56:21,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:21,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 168d6c8e660cdb03a471b9cc5e7a3e2b: 2023-07-19 05:56:21,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689746125590.168d6c8e660cdb03a471b9cc5e7a3e2b. 2023-07-19 05:56:21,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ecfca84342d05307f433bda44290dd5a, disabling compactions & flushes 2023-07-19 05:56:21,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:21,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:21,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. after waiting 0 ms 2023-07-19 05:56:21,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:21,938 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-19 05:56:21,939 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-19 05:56:21,939 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-19 05:56:21,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-19 05:56:21,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/hbase/namespace/ecfca84342d05307f433bda44290dd5a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-19 05:56:21,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:21,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ecfca84342d05307f433bda44290dd5a: 2023-07-19 05:56:21,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689746122929.ecfca84342d05307f433bda44290dd5a. 2023-07-19 05:56:21,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2054d9ec275367d29389fbedb9c09867, disabling compactions & flushes 2023-07-19 05:56:21,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:21,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:21,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. after waiting 0 ms 2023-07-19 05:56:21,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:21,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2054d9ec275367d29389fbedb9c09867 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-19 05:56:21,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/.tmp/cf1/c434a015ce1248c894dba67d410a0d9c 2023-07-19 05:56:21,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/.tmp/cf1/c434a015ce1248c894dba67d410a0d9c as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/c434a015ce1248c894dba67d410a0d9c 2023-07-19 05:56:21,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/cf1/c434a015ce1248c894dba67d410a0d9c, entries=100, sequenceid=21, filesize=8.3 K 2023-07-19 05:56:21,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 2054d9ec275367d29389fbedb9c09867 in 24ms, sequenceid=21, compaction requested=true 2023-07-19 05:56:21,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/data/default/t1_syncup/2054d9ec275367d29389fbedb9c09867/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-19 05:56:21,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:21,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2054d9ec275367d29389fbedb9c09867: 2023-07-19 05:56:21,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689746124972.2054d9ec275367d29389fbedb9c09867. 2023-07-19 05:56:22,058 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44553,1689746165143; all regions closed. 2023-07-19 05:56:22,063 DEBUG [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:56:22,063 INFO [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44553%2C1689746165143.meta:.meta(num 1689746165640) 2023-07-19 05:56:22,067 DEBUG [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:56:22,067 INFO [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44553%2C1689746165143.jenkins-hbase4.apache.org%2C44553%2C1689746165143.regiongroup-0:(num 1689746165520) 2023-07-19 05:56:22,072 DEBUG [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/oldWALs 2023-07-19 05:56:22,072 INFO [RS:0;jenkins-hbase4:44553] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44553%2C1689746165143.jenkins-hbase4.apache.org%2C44553%2C1689746165143.regiongroup-1:(num 1689746170180) 2023-07-19 05:56:22,072 DEBUG [RS:0;jenkins-hbase4:44553] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:22,072 INFO [RS:0;jenkins-hbase4:44553] regionserver.LeaseManager(133): Closed leases 2023-07-19 05:56:22,072 INFO [RS:0;jenkins-hbase4:44553] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-19 05:56:22,072 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:56:22,072 DEBUG [RS:0;jenkins-hbase4:44553] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x756a878c to 127.0.0.1:65243 2023-07-19 05:56:22,072 DEBUG [RS:0;jenkins-hbase4:44553] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:22,073 INFO [RS:0;jenkins-hbase4:44553] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44553 2023-07-19 05:56:22,077 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,44553,1689746165143 2023-07-19 05:56:22,077 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:22,077 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c0002a, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:22,077 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:22,077 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c00027, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:22,077 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017c2601c0003d, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-19 05:56:22,078 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:22,078 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:22,077 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-19 05:56:22,078 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44553,1689746165143] 2023-07-19 05:56:22,078 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44553,1689746165143; numProcessing=1 2023-07-19 05:56:22,080 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,44553,1689746165143 already deleted, retry=false 2023-07-19 05:56:22,081 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44553,1689746165143 expired; onlineServers=0 2023-07-19 05:56:22,081 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41737,1689746165120' ***** 2023-07-19 05:56:22,081 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-19 05:56:22,081 DEBUG [M:0;jenkins-hbase4:41737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5036f581, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-19 05:56:22,081 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41737,1689746165120 2023-07-19 05:56:22,081 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41737,1689746165120; all regions closed. 2023-07-19 05:56:22,081 DEBUG [M:0;jenkins-hbase4:41737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-19 05:56:22,081 DEBUG [M:0;jenkins-hbase4:41737] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-19 05:56:22,081 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-19 05:56:22,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746165315] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689746165315,5,FailOnTimeoutGroup] 2023-07-19 05:56:22,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746165315] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689746165315,5,FailOnTimeoutGroup] 2023-07-19 05:56:22,081 DEBUG [M:0;jenkins-hbase4:41737] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-19 05:56:22,083 INFO [M:0;jenkins-hbase4:41737] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-19 05:56:22,083 INFO [M:0;jenkins-hbase4:41737] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-19 05:56:22,083 INFO [M:0;jenkins-hbase4:41737] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-19 05:56:22,083 DEBUG [M:0;jenkins-hbase4:41737] master.HMaster(1512): Stopping service threads 2023-07-19 05:56:22,083 INFO [M:0;jenkins-hbase4:41737] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-19 05:56:22,084 ERROR [M:0;jenkins-hbase4:41737] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-19 05:56:22,084 INFO [M:0;jenkins-hbase4:41737] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-19 05:56:22,084 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-19 05:56:22,084 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-19 05:56:22,084 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-19 05:56:22,084 DEBUG [M:0;jenkins-hbase4:41737] zookeeper.ZKUtil(398): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-19 05:56:22,084 WARN [M:0;jenkins-hbase4:41737] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-19 05:56:22,084 INFO [M:0;jenkins-hbase4:41737] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-19 05:56:22,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-19 05:56:22,085 INFO [M:0;jenkins-hbase4:41737] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-19 05:56:22,086 DEBUG [M:0;jenkins-hbase4:41737] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-19 05:56:22,086 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:22,086 DEBUG [M:0;jenkins-hbase4:41737] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:22,086 DEBUG [M:0;jenkins-hbase4:41737] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-19 05:56:22,086 DEBUG [M:0;jenkins-hbase4:41737] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:22,086 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-19 05:56:22,099 INFO [M:0;jenkins-hbase4:41737] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=310 (bloomFilter=true), to=hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1d1fc31c4beb439cbccaf60bcf64fd12 2023-07-19 05:56:22,104 DEBUG [M:0;jenkins-hbase4:41737] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1d1fc31c4beb439cbccaf60bcf64fd12 as hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1d1fc31c4beb439cbccaf60bcf64fd12 2023-07-19 05:56:22,109 INFO [M:0;jenkins-hbase4:41737] regionserver.HStore(1080): Added hdfs://localhost:33839/user/jenkins/test-data/df7e8dd8-a751-9b65-09ee-85f89fd1b881/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1d1fc31c4beb439cbccaf60bcf64fd12, entries=9, sequenceid=310, filesize=5.5 K 2023-07-19 05:56:22,110 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=310, compaction requested=true 2023-07-19 05:56:22,110 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-19 05:56:22,111 DEBUG [M:0;jenkins-hbase4:41737] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-19 05:56:22,113 INFO [M:0;jenkins-hbase4:41737] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-19 05:56:22,113 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-19 05:56:22,114 INFO [M:0;jenkins-hbase4:41737] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41737 2023-07-19 05:56:22,117 DEBUG [M:0;jenkins-hbase4:41737] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,41737,1689746165120 already deleted, retry=false 2023-07-19 05:56:22,248 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:22,248 INFO [M:0;jenkins-hbase4:41737] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41737,1689746165120; zookeeper connection closed. 2023-07-19 05:56:22,249 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): master:41737-0x1017c2601c00033, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:22,349 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:22,349 INFO [RS:0;jenkins-hbase4:44553] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44553,1689746165143; zookeeper connection closed. 2023-07-19 05:56:22,349 DEBUG [Listener at localhost/45003-EventThread] zookeeper.ZKWatcher(600): regionserver:44553-0x1017c2601c00034, quorum=127.0.0.1:65243, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-19 05:56:22,349 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@62af335b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@62af335b 2023-07-19 05:56:22,349 INFO [Listener at localhost/45003] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-19 05:56:22,350 WARN [Listener at localhost/45003] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-19 05:56:22,354 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:22,460 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-19 05:56:22,460 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1616275405-172.31.14.131-1689746120389 (Datanode Uuid 7da89a07-25bf-4d11-b4fa-0f0716110233) service to localhost/127.0.0.1:33839 2023-07-19 05:56:22,462 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data7/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,462 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data8/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,465 WARN [Listener at localhost/45003] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-19 05:56:22,468 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:22,573 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-19 05:56:22,573 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1616275405-172.31.14.131-1689746120389 (Datanode Uuid a5183c54-915c-49cb-a0fd-a74b2feb6fb4) service to localhost/127.0.0.1:33839 2023-07-19 05:56:22,574 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data5/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,574 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data6/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,577 WARN [Listener at localhost/45003] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-19 05:56:22,580 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:22,684 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-19 05:56:22,684 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1616275405-172.31.14.131-1689746120389 (Datanode Uuid 5e7adc69-4203-4bea-b585-ecb199f232f2) service to localhost/127.0.0.1:33839 2023-07-19 05:56:22,684 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data3/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,685 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data4/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,686 WARN [Listener at localhost/45003] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-19 05:56:22,689 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:22,794 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-19 05:56:22,794 WARN [BP-1616275405-172.31.14.131-1689746120389 heartbeating to localhost/127.0.0.1:33839] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1616275405-172.31.14.131-1689746120389 (Datanode Uuid 31c61bf0-0d47-4d7e-8864-33d9f690413c) service to localhost/127.0.0.1:33839 2023-07-19 05:56:22,795 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data1/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,795 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7454bb72-84e6-c86c-5fb3-5308e1fe9488/cluster_a04a6bde-4817-e640-9a7c-37e1e8bc8e9e/dfs/data/data2/current/BP-1616275405-172.31.14.131-1689746120389] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:22,826 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:23,002 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-19 05:56:23,002 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-19 05:56:23,003 WARN [Listener at localhost/45003] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-19 05:56:23,005 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:23,106 WARN [BP-22750715-172.31.14.131-1689746113329 heartbeating to localhost/127.0.0.1:46083] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-19 05:56:23,106 WARN [BP-22750715-172.31.14.131-1689746113329 heartbeating to localhost/127.0.0.1:46083] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-22750715-172.31.14.131-1689746113329 (Datanode Uuid 32f7061d-5342-4160-b2b9-ba7bbf92aa63) service to localhost/127.0.0.1:46083 2023-07-19 05:56:23,107 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/dfs/data/data3/current/BP-22750715-172.31.14.131-1689746113329] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:23,107 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/dfs/data/data4/current/BP-22750715-172.31.14.131-1689746113329] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:23,108 WARN [Listener at localhost/45003] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-19 05:56:23,110 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:23,210 WARN [BP-22750715-172.31.14.131-1689746113329 heartbeating to localhost/127.0.0.1:46083] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-19 05:56:23,211 WARN [BP-22750715-172.31.14.131-1689746113329 heartbeating to localhost/127.0.0.1:46083] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-22750715-172.31.14.131-1689746113329 (Datanode Uuid 65e74bcf-2b26-4b98-aada-29a63cab5a77) service to localhost/127.0.0.1:46083 2023-07-19 05:56:23,211 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/dfs/data/data1/current/BP-22750715-172.31.14.131-1689746113329] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:23,212 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0ee1073-f121-c9e0-ee90-785787f05d0c/cluster_0a51169e-e24f-9d46-32c4-1d8dccd78986/dfs/data/data2/current/BP-22750715-172.31.14.131-1689746113329] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-19 05:56:23,219 INFO [Listener at localhost/45003] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-19 05:56:23,338 INFO [Listener at localhost/45003] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-19 05:56:23,368 INFO [Listener at localhost/45003] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-19 05:56:23,389 INFO [Listener at localhost/45003] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=188 (was 10) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:46083 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x7c65aeff-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x22818760-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42035,1689746157262-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (684122768) connection to localhost/127.0.0.1:33839 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:46083 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x67e04b10-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (684122768) connection to localhost/127.0.0.1:33839 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: Listener at localhost/45003 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (684122768) connection to localhost/127.0.0.1:33839 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@3aa01244 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x67e04b10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/998978330.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:46083 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x67e04b10-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x13ba4784-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:46083 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x23ab8a15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/998978330.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x23ab8a15-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x22818760-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x13ba4784 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/998978330.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x7c65aeff-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:33839 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x7c65aeff sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/998978330.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x22818760 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/998978330.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43215,1689746117482-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (684122768) connection to localhost/127.0.0.1:46083 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x23ab8a15-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (684122768) connection to localhost/127.0.0.1:46083 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65243@0x13ba4784-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,43759,1689746117439-SendThread(127.0.0.1:65243) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=663 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=168 (was 225), ProcessCount=171 (was 174), AvailableMemoryMB=5489 (was 7097)