2023-07-18 03:55:23,460 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf 2023-07-18 03:55:23,470 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-log-dir so I do NOT create it in target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399 2023-07-18 03:55:23,470 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-log-dir Erasing configuration value by system value. 2023-07-18 03:55:23,470 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-tmp-dir so I do NOT create it in target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399 2023-07-18 03:55:23,471 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-18 03:55:23,471 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399 2023-07-18 03:55:23,490 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-18 03:55:23,526 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.TestReplicationSyncUpTool#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=224, ProcessCount=172, AvailableMemoryMB=7009 2023-07-18 03:55:23,537 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-log-dir so I do NOT create it in target/test-data/a26c9689-639b-f3d8-1f6c-2f6e62a67f47 2023-07-18 03:55:23,537 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-log-dir Erasing configuration value by system value. 2023-07-18 03:55:23,537 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-tmp-dir so I do NOT create it in target/test-data/a26c9689-639b-f3d8-1f6c-2f6e62a67f47 2023-07-18 03:55:23,538 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-18 03:55:23,538 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a26c9689-639b-f3d8-1f6c-2f6e62a67f47 2023-07-18 03:55:23,545 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-log-dir so I do NOT create it in target/test-data/e4742bef-4533-432b-4e72-15ad312c5303 2023-07-18 03:55:23,545 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-log-dir Erasing configuration value by system value. 2023-07-18 03:55:23,545 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-tmp-dir so I do NOT create it in target/test-data/e4742bef-4533-432b-4e72-15ad312c5303 2023-07-18 03:55:23,545 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-18 03:55:23,545 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e4742bef-4533-432b-4e72-15ad312c5303 2023-07-18 03:55:23,549 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8, deleteOnExit=true 2023-07-18 03:55:23,676 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/zookeeper_0, clientPort=60939, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-18 03:55:23,692 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=60939 2023-07-18 03:55:23,696 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-18 03:55:23,696 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-18 03:55:23,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/test.cache.data in system properties and HBase conf 2023-07-18 03:55:23,697 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop.tmp.dir in system properties and HBase conf 2023-07-18 03:55:23,697 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/hadoop.log.dir in system properties and HBase conf 2023-07-18 03:55:23,697 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-18 03:55:23,698 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-18 03:55:23,698 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-18 03:55:23,798 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-18 03:55:24,198 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-18 03:55:24,201 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-18 03:55:24,202 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-18 03:55:24,202 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-18 03:55:24,203 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 03:55:24,203 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-18 03:55:24,203 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-18 03:55:24,204 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 03:55:24,204 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 03:55:24,204 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-18 03:55:24,205 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/nfs.dump.dir in system properties and HBase conf 2023-07-18 03:55:24,205 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/java.io.tmpdir in system properties and HBase conf 2023-07-18 03:55:24,205 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 03:55:24,206 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-18 03:55:24,206 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-18 03:55:24,790 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 03:55:24,796 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 03:55:25,125 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-18 03:55:25,263 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-18 03:55:25,284 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:25,329 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:25,364 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/java.io.tmpdir/Jetty_localhost_40383_hdfs____.i7pyhw/webapp 2023-07-18 03:55:25,511 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40383 2023-07-18 03:55:25,529 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 03:55:25,529 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 03:55:26,070 WARN [Listener at localhost/40155] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:26,141 WARN [Listener at localhost/40155] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 03:55:26,159 WARN [Listener at localhost/40155] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:26,165 INFO [Listener at localhost/40155] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:26,169 INFO [Listener at localhost/40155] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/java.io.tmpdir/Jetty_localhost_36909_datanode____.l8tl96/webapp 2023-07-18 03:55:26,266 INFO [Listener at localhost/40155] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36909 2023-07-18 03:55:26,580 WARN [Listener at localhost/34475] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:26,600 WARN [Listener at localhost/34475] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 03:55:26,609 WARN [Listener at localhost/34475] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:26,611 INFO [Listener at localhost/34475] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:26,623 INFO [Listener at localhost/34475] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/java.io.tmpdir/Jetty_localhost_39063_datanode____.7w4x5e/webapp 2023-07-18 03:55:26,735 INFO [Listener at localhost/34475] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39063 2023-07-18 03:55:26,745 WARN [Listener at localhost/38987] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:27,163 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf560a8e48946d84f: Processing first storage report for DS-5dcf1289-5fbd-4710-b303-15a85102257f from datanode 18443c7a-db9c-4626-bd93-45d3820bdeb4 2023-07-18 03:55:27,164 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf560a8e48946d84f: from storage DS-5dcf1289-5fbd-4710-b303-15a85102257f node DatanodeRegistration(127.0.0.1:42603, datanodeUuid=18443c7a-db9c-4626-bd93-45d3820bdeb4, infoPort=44729, infoSecurePort=0, ipcPort=38987, storageInfo=lv=-57;cid=testClusterID;nsid=864352436;c=1689652524884), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-18 03:55:27,165 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x93023e67164554a2: Processing first storage report for DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235 from datanode 091fa510-0dd7-4ae4-a681-f35382dac693 2023-07-18 03:55:27,165 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x93023e67164554a2: from storage DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235 node DatanodeRegistration(127.0.0.1:38001, datanodeUuid=091fa510-0dd7-4ae4-a681-f35382dac693, infoPort=33249, infoSecurePort=0, ipcPort=34475, storageInfo=lv=-57;cid=testClusterID;nsid=864352436;c=1689652524884), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:27,165 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf560a8e48946d84f: Processing first storage report for DS-ef6eceaa-adb8-42a6-8cd0-8e7ef46a3142 from datanode 18443c7a-db9c-4626-bd93-45d3820bdeb4 2023-07-18 03:55:27,166 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf560a8e48946d84f: from storage DS-ef6eceaa-adb8-42a6-8cd0-8e7ef46a3142 node DatanodeRegistration(127.0.0.1:42603, datanodeUuid=18443c7a-db9c-4626-bd93-45d3820bdeb4, infoPort=44729, infoSecurePort=0, ipcPort=38987, storageInfo=lv=-57;cid=testClusterID;nsid=864352436;c=1689652524884), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-18 03:55:27,167 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x93023e67164554a2: Processing first storage report for DS-77ab0ded-351b-4ed6-98b3-65bd92fc44e5 from datanode 091fa510-0dd7-4ae4-a681-f35382dac693 2023-07-18 03:55:27,167 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x93023e67164554a2: from storage DS-77ab0ded-351b-4ed6-98b3-65bd92fc44e5 node DatanodeRegistration(127.0.0.1:38001, datanodeUuid=091fa510-0dd7-4ae4-a681-f35382dac693, infoPort=33249, infoSecurePort=0, ipcPort=34475, storageInfo=lv=-57;cid=testClusterID;nsid=864352436;c=1689652524884), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:27,220 DEBUG [Listener at localhost/38987] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf 2023-07-18 03:55:27,228 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:27,230 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:27,928 INFO [Listener at localhost/38987] util.FSUtils(471): Created version file at hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26 with version=8 2023-07-18 03:55:27,928 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/hbase-staging 2023-07-18 03:55:28,307 INFO [Listener at localhost/38987] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-18 03:55:28,914 INFO [Listener at localhost/38987] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 03:55:28,959 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:28,960 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:28,961 INFO [Listener at localhost/38987] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:28,961 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:28,961 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:29,129 INFO [Listener at localhost/38987] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:29,200 DEBUG [Listener at localhost/38987] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-18 03:55:29,304 INFO [Listener at localhost/38987] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41639 2023-07-18 03:55:29,314 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,316 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,337 INFO [Listener at localhost/38987] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41639 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:29,380 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:416390x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:29,382 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41639-0x1017691f55f0000 connected 2023-07-18 03:55:29,407 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:55:29,407 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:55:29,411 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 03:55:29,418 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41639 2023-07-18 03:55:29,419 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41639 2023-07-18 03:55:29,420 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41639 2023-07-18 03:55:29,420 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41639 2023-07-18 03:55:29,420 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41639 2023-07-18 03:55:29,425 INFO [Listener at localhost/38987] master.HMaster(444): hbase.rootdir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26, hbase.cluster.distributed=false 2023-07-18 03:55:29,502 INFO [Listener at localhost/38987] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 03:55:29,502 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:29,502 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:29,502 INFO [Listener at localhost/38987] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:29,503 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:29,503 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:29,508 INFO [Listener at localhost/38987] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:29,511 INFO [Listener at localhost/38987] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36729 2023-07-18 03:55:29,513 INFO [Listener at localhost/38987] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:29,520 DEBUG [Listener at localhost/38987] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:29,521 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,523 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,524 INFO [Listener at localhost/38987] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36729 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:29,528 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:367290x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:29,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36729-0x1017691f55f0001 connected 2023-07-18 03:55:29,528 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:55:29,530 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:55:29,531 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 03:55:29,531 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36729 2023-07-18 03:55:29,531 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36729 2023-07-18 03:55:29,532 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36729 2023-07-18 03:55:29,532 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36729 2023-07-18 03:55:29,533 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36729 2023-07-18 03:55:29,545 INFO [Listener at localhost/38987] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 03:55:29,545 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:29,545 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:29,546 INFO [Listener at localhost/38987] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:29,546 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:29,546 INFO [Listener at localhost/38987] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:29,546 INFO [Listener at localhost/38987] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:29,547 INFO [Listener at localhost/38987] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42337 2023-07-18 03:55:29,548 INFO [Listener at localhost/38987] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:29,549 DEBUG [Listener at localhost/38987] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:29,550 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,552 INFO [Listener at localhost/38987] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,554 INFO [Listener at localhost/38987] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42337 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:29,557 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:423370x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:29,558 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): regionserver:423370x0, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:55:29,559 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): regionserver:423370x0, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:55:29,560 DEBUG [Listener at localhost/38987] zookeeper.ZKUtil(164): regionserver:423370x0, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 03:55:29,560 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42337 2023-07-18 03:55:29,561 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42337 2023-07-18 03:55:29,562 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42337 2023-07-18 03:55:29,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42337-0x1017691f55f0002 connected 2023-07-18 03:55:29,562 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42337 2023-07-18 03:55:29,563 DEBUG [Listener at localhost/38987] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42337 2023-07-18 03:55:29,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:29,573 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 03:55:29,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:29,595 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 03:55:29,595 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 03:55:29,595 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 03:55:29,595 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:29,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 03:55:29,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 03:55:29,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,41639,1689652528089 from backup master directory 2023-07-18 03:55:29,609 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:29,609 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 03:55:29,610 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:29,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:29,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-18 03:55:29,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-18 03:55:29,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/hbase.id with ID: 0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 2023-07-18 03:55:29,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:29,776 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:29,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3886878b to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:29,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a085707, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:29,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:29,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 03:55:29,892 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-18 03:55:29,892 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-18 03:55:29,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:29,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:29,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:29,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store-tmp 2023-07-18 03:55:29,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:29,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:55:29,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:29,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:29,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:55:29,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:29,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:29,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:29,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:29,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41639%2C1689652528089, suffix=, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/oldWALs, maxLogs=10 2023-07-18 03:55:30,039 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:55:30,039 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:55:30,047 DEBUG [RS-EventLoopGroup-4-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:30,112 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 2023-07-18 03:55:30,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK], DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK]] 2023-07-18 03:55:30,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:30,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:30,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:30,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:30,183 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:30,192 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 03:55:30,216 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 03:55:30,228 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:30,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:30,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:30,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:30,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:30,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59776325, jitterRate=-0.1092633455991745}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:55:30,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:30,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 03:55:30,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 03:55:30,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 03:55:30,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 03:55:30,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-18 03:55:30,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-07-18 03:55:30,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 03:55:30,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-18 03:55:30,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-18 03:55:30,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 03:55:30,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 03:55:30,377 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-18 03:55:30,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 03:55:30,390 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-18 03:55:30,393 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:30,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-18 03:55:30,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-18 03:55:30,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-18 03:55:30,414 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 03:55:30,414 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 03:55:30,414 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 03:55:30,414 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:30,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41639,1689652528089, sessionid=0x1017691f55f0000, setting cluster-up flag (Was=false) 2023-07-18 03:55:30,432 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:30,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-18 03:55:30,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:30,447 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:30,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-18 03:55:30,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:30,456 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.hbase-snapshot/.tmp 2023-07-18 03:55:30,466 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(951): ClusterId : 0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 2023-07-18 03:55:30,466 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(951): ClusterId : 0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 2023-07-18 03:55:30,471 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:30,471 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:30,478 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:30,478 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:30,478 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:30,478 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:30,481 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:30,481 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:30,483 DEBUG [RS:1;jenkins-hbase4:42337] zookeeper.ReadOnlyZKClient(139): Connect 0x23232f5a to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:30,483 DEBUG [RS:0;jenkins-hbase4:36729] zookeeper.ReadOnlyZKClient(139): Connect 0x1b39d69d to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:30,488 DEBUG [RS:1;jenkins-hbase4:42337] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2de4d54e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:30,488 DEBUG [RS:0;jenkins-hbase4:36729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15fe3b8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:30,488 DEBUG [RS:1;jenkins-hbase4:42337] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29db442a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:30,489 DEBUG [RS:0;jenkins-hbase4:36729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11c1bded, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:30,510 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:42337 2023-07-18 03:55:30,514 INFO [RS:1;jenkins-hbase4:42337] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:30,514 INFO [RS:1;jenkins-hbase4:42337] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:30,514 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:30,514 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36729 2023-07-18 03:55:30,515 INFO [RS:0;jenkins-hbase4:36729] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:30,515 INFO [RS:0;jenkins-hbase4:36729] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:30,515 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:30,517 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41639,1689652528089 with isa=jenkins-hbase4.apache.org/172.31.14.131:42337, startcode=1689652529544 2023-07-18 03:55:30,517 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41639,1689652528089 with isa=jenkins-hbase4.apache.org/172.31.14.131:36729, startcode=1689652529501 2023-07-18 03:55:30,535 DEBUG [RS:1;jenkins-hbase4:42337] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:30,535 DEBUG [RS:0;jenkins-hbase4:36729] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:30,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:30,590 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48055, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:30,590 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45433, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:30,603 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41639] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-18 03:55:30,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:30,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:30,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:30,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:30,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 03:55:30,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:30,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689652560610 2023-07-18 03:55:30,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 03:55:30,615 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41639] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-18 03:55:30,616 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:30,616 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-18 03:55:30,623 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:30,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 03:55:30,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 03:55:30,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 03:55:30,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 03:55:30,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 03:55:30,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 03:55:30,637 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(2830): Master is not running yet 2023-07-18 03:55:30,637 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(2830): Master is not running yet 2023-07-18 03:55:30,637 WARN [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-18 03:55:30,637 WARN [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-18 03:55:30,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 03:55:30,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 03:55:30,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 03:55:30,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 03:55:30,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652530649,5,FailOnTimeoutGroup] 2023-07-18 03:55:30,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652530650,5,FailOnTimeoutGroup] 2023-07-18 03:55:30,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 03:55:30,653 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,653 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,661 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:30,663 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:30,663 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26 2023-07-18 03:55:30,681 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:30,684 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:55:30,687 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info 2023-07-18 03:55:30,688 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:55:30,689 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:30,689 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:55:30,692 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:30,693 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:55:30,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:30,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:55:30,697 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table 2023-07-18 03:55:30,697 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:55:30,698 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:30,700 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740 2023-07-18 03:55:30,701 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740 2023-07-18 03:55:30,704 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:55:30,707 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:55:30,711 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:30,712 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73273465, jitterRate=0.09185971319675446}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:55:30,712 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:55:30,713 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:55:30,713 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:55:30,713 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:55:30,713 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:55:30,713 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:55:30,714 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:55:30,714 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:55:30,719 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:30,719 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-18 03:55:30,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 03:55:30,738 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41639,1689652528089 with isa=jenkins-hbase4.apache.org/172.31.14.131:36729, startcode=1689652529501 2023-07-18 03:55:30,738 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41639,1689652528089 with isa=jenkins-hbase4.apache.org/172.31.14.131:42337, startcode=1689652529544 2023-07-18 03:55:30,739 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 03:55:30,742 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-18 03:55:30,744 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41639] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,750 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41639] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,751 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26 2023-07-18 03:55:30,752 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40155 2023-07-18 03:55:30,752 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:30,752 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26 2023-07-18 03:55:30,752 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40155 2023-07-18 03:55:30,752 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:30,759 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:55:30,760 DEBUG [RS:0;jenkins-hbase4:36729] zookeeper.ZKUtil(162): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,760 WARN [RS:0;jenkins-hbase4:36729] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:30,760 INFO [RS:0;jenkins-hbase4:36729] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:30,760 DEBUG [RS:1;jenkins-hbase4:42337] zookeeper.ZKUtil(162): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,760 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,760 WARN [RS:1;jenkins-hbase4:42337] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:30,761 INFO [RS:1;jenkins-hbase4:42337] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:30,761 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,762 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42337,1689652529544] 2023-07-18 03:55:30,762 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36729,1689652529501] 2023-07-18 03:55:30,771 DEBUG [RS:1;jenkins-hbase4:42337] zookeeper.ZKUtil(162): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,772 DEBUG [RS:1;jenkins-hbase4:42337] zookeeper.ZKUtil(162): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,775 DEBUG [RS:0;jenkins-hbase4:36729] zookeeper.ZKUtil(162): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,776 DEBUG [RS:0;jenkins-hbase4:36729] zookeeper.ZKUtil(162): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,783 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:30,783 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:30,794 INFO [RS:0;jenkins-hbase4:36729] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:30,794 INFO [RS:1;jenkins-hbase4:42337] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:30,815 INFO [RS:1;jenkins-hbase4:42337] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:30,815 INFO [RS:0;jenkins-hbase4:36729] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:30,819 INFO [RS:1;jenkins-hbase4:42337] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:30,819 INFO [RS:0;jenkins-hbase4:36729] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:30,820 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,820 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,821 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:30,821 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:30,828 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,828 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,828 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,828 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,829 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,830 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,830 DEBUG [RS:1;jenkins-hbase4:42337] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,830 DEBUG [RS:0;jenkins-hbase4:36729] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:30,831 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,831 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,831 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,831 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,831 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,831 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,848 INFO [RS:0;jenkins-hbase4:36729] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:30,848 INFO [RS:1;jenkins-hbase4:42337] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:30,851 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42337,1689652529544-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,851 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36729,1689652529501-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:30,869 INFO [RS:0;jenkins-hbase4:36729] regionserver.Replication(203): jenkins-hbase4.apache.org,36729,1689652529501 started 2023-07-18 03:55:30,869 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36729,1689652529501, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36729, sessionid=0x1017691f55f0001 2023-07-18 03:55:30,870 INFO [RS:1;jenkins-hbase4:42337] regionserver.Replication(203): jenkins-hbase4.apache.org,42337,1689652529544 started 2023-07-18 03:55:30,870 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:30,871 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42337,1689652529544, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42337, sessionid=0x1017691f55f0002 2023-07-18 03:55:30,871 DEBUG [RS:0;jenkins-hbase4:36729] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,871 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:30,871 DEBUG [RS:0;jenkins-hbase4:36729] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36729,1689652529501' 2023-07-18 03:55:30,871 DEBUG [RS:1;jenkins-hbase4:42337] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,871 DEBUG [RS:0;jenkins-hbase4:36729] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-18 03:55:30,872 DEBUG [RS:1;jenkins-hbase4:42337] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42337,1689652529544' 2023-07-18 03:55:30,872 DEBUG [RS:1;jenkins-hbase4:42337] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-18 03:55:30,873 DEBUG [RS:1;jenkins-hbase4:42337] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-18 03:55:30,873 DEBUG [RS:0;jenkins-hbase4:36729] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-18 03:55:30,873 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:30,873 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:30,873 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:30,873 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:30,874 DEBUG [RS:1;jenkins-hbase4:42337] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:30,874 DEBUG [RS:0;jenkins-hbase4:36729] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:30,874 DEBUG [RS:1;jenkins-hbase4:42337] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42337,1689652529544' 2023-07-18 03:55:30,874 DEBUG [RS:1;jenkins-hbase4:42337] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-18 03:55:30,874 DEBUG [RS:0;jenkins-hbase4:36729] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36729,1689652529501' 2023-07-18 03:55:30,874 DEBUG [RS:0;jenkins-hbase4:36729] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-18 03:55:30,875 DEBUG [RS:1;jenkins-hbase4:42337] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-18 03:55:30,875 DEBUG [RS:0;jenkins-hbase4:36729] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-18 03:55:30,875 DEBUG [RS:1;jenkins-hbase4:42337] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:30,875 INFO [RS:1;jenkins-hbase4:42337] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:30,875 INFO [RS:1;jenkins-hbase4:42337] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:30,876 DEBUG [RS:0;jenkins-hbase4:36729] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:30,876 INFO [RS:0;jenkins-hbase4:36729] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:30,876 INFO [RS:0;jenkins-hbase4:36729] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:30,894 DEBUG [jenkins-hbase4:41639] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-18 03:55:30,897 DEBUG [jenkins-hbase4:41639] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:30,903 DEBUG [jenkins-hbase4:41639] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:30,903 DEBUG [jenkins-hbase4:41639] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:30,903 DEBUG [jenkins-hbase4:41639] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:30,906 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36729,1689652529501, state=OPENING 2023-07-18 03:55:30,921 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-18 03:55:30,922 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:30,923 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 03:55:30,926 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36729,1689652529501}] 2023-07-18 03:55:30,987 INFO [RS:1;jenkins-hbase4:42337] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42337%2C1689652529544, suffix=, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs, maxLogs=10 2023-07-18 03:55:30,987 INFO [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36729%2C1689652529501, suffix=, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs, maxLogs=10 2023-07-18 03:55:31,011 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:55:31,013 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:55:31,014 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:55:31,014 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:55:31,020 INFO [RS:1;jenkins-hbase4:42337] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 2023-07-18 03:55:31,020 INFO [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992 2023-07-18 03:55:31,020 DEBUG [RS:1;jenkins-hbase4:42337] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK], DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK]] 2023-07-18 03:55:31,020 DEBUG [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK], DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK]] 2023-07-18 03:55:31,113 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:31,116 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:31,119 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48454, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:31,131 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 03:55:31,131 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:31,135 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36729%2C1689652529501.meta, suffix=.meta, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs, maxLogs=10 2023-07-18 03:55:31,158 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:55:31,159 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:55:31,167 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.meta.1689652531137.meta 2023-07-18 03:55:31,170 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK], DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK]] 2023-07-18 03:55:31,171 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:31,173 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 03:55:31,189 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 03:55:31,215 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 03:55:31,220 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 03:55:31,220 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:31,220 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 03:55:31,220 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 03:55:31,223 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:55:31,225 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info 2023-07-18 03:55:31,225 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info 2023-07-18 03:55:31,226 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:55:31,227 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:31,227 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:55:31,228 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:31,229 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:31,229 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:55:31,231 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:31,232 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:55:31,233 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table 2023-07-18 03:55:31,233 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table 2023-07-18 03:55:31,233 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:55:31,234 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:31,236 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740 2023-07-18 03:55:31,239 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740 2023-07-18 03:55:31,243 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:55:31,250 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:55:31,252 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71231151, jitterRate=0.06142686307430267}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:55:31,252 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:55:31,264 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689652531105 2023-07-18 03:55:31,285 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 03:55:31,286 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 03:55:31,287 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36729,1689652529501, state=OPEN 2023-07-18 03:55:31,290 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-18 03:55:31,290 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 03:55:31,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-18 03:55:31,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36729,1689652529501 in 365 msec 2023-07-18 03:55:31,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-18 03:55:31,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 569 msec 2023-07-18 03:55:31,307 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 794 msec 2023-07-18 03:55:31,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689652531308, completionTime=-1 2023-07-18 03:55:31,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-18 03:55:31,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 03:55:31,389 DEBUG [hconnection-0x7b1bd683-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:31,392 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48468, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:31,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-18 03:55:31,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689652591415 2023-07-18 03:55:31,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689652651415 2023-07-18 03:55:31,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 107 msec 2023-07-18 03:55:31,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41639,1689652528089-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:31,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41639,1689652528089-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:31,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41639,1689652528089-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:31,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41639, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:31,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:31,457 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-18 03:55:31,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-18 03:55:31,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:31,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-18 03:55:31,481 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 03:55:31,484 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 03:55:31,508 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,510 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 empty. 2023-07-18 03:55:31,511 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,511 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-18 03:55:31,538 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:31,540 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 12eb339fe9277c226d5f3d9d08310c02, NAME => 'hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp 2023-07-18 03:55:31,554 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:31,555 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 12eb339fe9277c226d5f3d9d08310c02, disabling compactions & flushes 2023-07-18 03:55:31,555 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,555 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,555 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. after waiting 0 ms 2023-07-18 03:55:31,555 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,555 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,555 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 12eb339fe9277c226d5f3d9d08310c02: 2023-07-18 03:55:31,559 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 03:55:31,575 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652531562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689652531562"}]},"ts":"1689652531562"} 2023-07-18 03:55:31,605 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 03:55:31,608 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 03:55:31,613 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652531608"}]},"ts":"1689652531608"} 2023-07-18 03:55:31,617 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-18 03:55:31,622 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:31,625 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:31,625 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:31,625 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:31,629 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN}] 2023-07-18 03:55:31,633 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN 2023-07-18 03:55:31,635 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36729,1689652529501; forceNewPlan=false, retain=false 2023-07-18 03:55:31,787 INFO [jenkins-hbase4:41639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 03:55:31,788 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=12eb339fe9277c226d5f3d9d08310c02, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:31,788 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652531788"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652531788"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652531788"}]},"ts":"1689652531788"} 2023-07-18 03:55:31,792 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 12eb339fe9277c226d5f3d9d08310c02, server=jenkins-hbase4.apache.org,36729,1689652529501}] 2023-07-18 03:55:31,952 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,953 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 12eb339fe9277c226d5f3d9d08310c02, NAME => 'hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:31,953 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,953 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:31,953 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,954 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,956 INFO [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,958 DEBUG [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info 2023-07-18 03:55:31,958 DEBUG [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info 2023-07-18 03:55:31,959 INFO [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12eb339fe9277c226d5f3d9d08310c02 columnFamilyName info 2023-07-18 03:55:31,960 INFO [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] regionserver.HStore(310): Store=12eb339fe9277c226d5f3d9d08310c02/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:31,961 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,962 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,967 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:31,970 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:31,971 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 12eb339fe9277c226d5f3d9d08310c02; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59249312, jitterRate=-0.11711645126342773}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:55:31,972 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 12eb339fe9277c226d5f3d9d08310c02: 2023-07-18 03:55:31,974 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02., pid=6, masterSystemTime=1689652531945 2023-07-18 03:55:31,978 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,978 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:31,979 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=12eb339fe9277c226d5f3d9d08310c02, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:31,979 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652531978"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652531978"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652531978"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652531978"}]},"ts":"1689652531978"} 2023-07-18 03:55:31,987 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-18 03:55:31,987 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 12eb339fe9277c226d5f3d9d08310c02, server=jenkins-hbase4.apache.org,36729,1689652529501 in 191 msec 2023-07-18 03:55:31,991 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-18 03:55:31,992 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN in 359 msec 2023-07-18 03:55:31,993 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 03:55:31,993 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652531993"}]},"ts":"1689652531993"} 2023-07-18 03:55:31,997 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-18 03:55:32,000 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 03:55:32,002 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 532 msec 2023-07-18 03:55:32,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-18 03:55:32,082 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-18 03:55:32,082 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:32,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-18 03:55:32,138 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-18 03:55:32,144 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-07-18 03:55:32,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-18 03:55:32,168 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-18 03:55:32,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-07-18 03:55:32,183 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-18 03:55:32,185 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-18 03:55:32,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.577sec 2023-07-18 03:55:32,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 03:55:32,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 03:55:32,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 03:55:32,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41639,1689652528089-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 03:55:32,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41639,1689652528089-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 03:55:32,202 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 03:55:32,275 DEBUG [Listener at localhost/38987] zookeeper.ReadOnlyZKClient(139): Connect 0x448b0fe6 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:32,281 DEBUG [Listener at localhost/38987] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f4388b9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:32,295 DEBUG [hconnection-0x435a5a3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:32,307 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48478, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:32,317 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:32,318 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-18 03:55:32,318 INFO [Listener at localhost/38987] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542, deleteOnExit=true 2023-07-18 03:55:32,318 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-18 03:55:32,329 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/test.cache.data in system properties and HBase conf 2023-07-18 03:55:32,329 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/hadoop.tmp.dir in system properties and HBase conf 2023-07-18 03:55:32,329 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/hadoop.log.dir in system properties and HBase conf 2023-07-18 03:55:32,329 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-18 03:55:32,329 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-18 03:55:32,329 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-18 03:55:32,329 DEBUG [Listener at localhost/38987] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-18 03:55:32,330 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-18 03:55:32,330 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-18 03:55:32,330 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-18 03:55:32,331 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 03:55:32,331 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-18 03:55:32,331 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-18 03:55:32,331 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 03:55:32,332 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 03:55:32,332 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-18 03:55:32,332 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/nfs.dump.dir in system properties and HBase conf 2023-07-18 03:55:32,332 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/java.io.tmpdir in system properties and HBase conf 2023-07-18 03:55:32,332 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 03:55:32,333 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-18 03:55:32,333 INFO [Listener at localhost/38987] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-18 03:55:32,348 WARN [Listener at localhost/38987] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 03:55:32,349 WARN [Listener at localhost/38987] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 03:55:32,401 WARN [Listener at localhost/38987] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:32,404 INFO [Listener at localhost/38987] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:32,412 INFO [Listener at localhost/38987] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/java.io.tmpdir/Jetty_localhost_36557_hdfs____.rqv20y/webapp 2023-07-18 03:55:32,525 INFO [Listener at localhost/38987] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36557 2023-07-18 03:55:32,538 WARN [Listener at localhost/38987] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 03:55:32,538 WARN [Listener at localhost/38987] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 03:55:32,604 WARN [Listener at localhost/36001] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:32,621 WARN [Listener at localhost/36001] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 03:55:32,623 WARN [Listener at localhost/36001] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:32,625 INFO [Listener at localhost/36001] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:32,632 INFO [Listener at localhost/36001] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/java.io.tmpdir/Jetty_localhost_34537_datanode____mlgy7z/webapp 2023-07-18 03:55:32,744 INFO [Listener at localhost/36001] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34537 2023-07-18 03:55:32,755 WARN [Listener at localhost/45343] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:32,825 WARN [Listener at localhost/45343] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 03:55:32,828 WARN [Listener at localhost/45343] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:32,829 INFO [Listener at localhost/45343] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:32,833 INFO [Listener at localhost/45343] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/java.io.tmpdir/Jetty_localhost_33697_datanode____5t0a57/webapp 2023-07-18 03:55:32,937 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x37c566c437a4353d: Processing first storage report for DS-55311c2a-76ef-4031-bb64-f570b49240d7 from datanode beba6500-c858-4d25-9c79-42e747dc8fe0 2023-07-18 03:55:32,937 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x37c566c437a4353d: from storage DS-55311c2a-76ef-4031-bb64-f570b49240d7 node DatanodeRegistration(127.0.0.1:34653, datanodeUuid=beba6500-c858-4d25-9c79-42e747dc8fe0, infoPort=46817, infoSecurePort=0, ipcPort=45343, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:32,937 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x37c566c437a4353d: Processing first storage report for DS-e4eb849e-99f3-41c0-ad1b-aefa9e3a9ada from datanode beba6500-c858-4d25-9c79-42e747dc8fe0 2023-07-18 03:55:32,937 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x37c566c437a4353d: from storage DS-e4eb849e-99f3-41c0-ad1b-aefa9e3a9ada node DatanodeRegistration(127.0.0.1:34653, datanodeUuid=beba6500-c858-4d25-9c79-42e747dc8fe0, infoPort=46817, infoSecurePort=0, ipcPort=45343, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:32,967 INFO [Listener at localhost/45343] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33697 2023-07-18 03:55:32,981 WARN [Listener at localhost/34347] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:33,033 WARN [Listener at localhost/34347] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 03:55:33,039 WARN [Listener at localhost/34347] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:33,041 INFO [Listener at localhost/34347] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:33,045 INFO [Listener at localhost/34347] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/java.io.tmpdir/Jetty_localhost_34727_datanode____9sgwma/webapp 2023-07-18 03:55:33,230 INFO [Listener at localhost/34347] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34727 2023-07-18 03:55:33,249 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x79daaf45ac197e67: Processing first storage report for DS-cfef0c64-8b80-4de2-8266-7d6085198f26 from datanode fedb5cbb-ab52-44f2-bc0e-eef7cdfba526 2023-07-18 03:55:33,249 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x79daaf45ac197e67: from storage DS-cfef0c64-8b80-4de2-8266-7d6085198f26 node DatanodeRegistration(127.0.0.1:45629, datanodeUuid=fedb5cbb-ab52-44f2-bc0e-eef7cdfba526, infoPort=39849, infoSecurePort=0, ipcPort=34347, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:33,249 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x79daaf45ac197e67: Processing first storage report for DS-c34b4495-ec25-4032-bed6-fbff67edae19 from datanode fedb5cbb-ab52-44f2-bc0e-eef7cdfba526 2023-07-18 03:55:33,249 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x79daaf45ac197e67: from storage DS-c34b4495-ec25-4032-bed6-fbff67edae19 node DatanodeRegistration(127.0.0.1:45629, datanodeUuid=fedb5cbb-ab52-44f2-bc0e-eef7cdfba526, infoPort=39849, infoSecurePort=0, ipcPort=34347, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:33,262 WARN [Listener at localhost/46647] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:33,330 WARN [Listener at localhost/46647] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 03:55:33,340 WARN [Listener at localhost/46647] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 03:55:33,341 INFO [Listener at localhost/46647] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 03:55:33,350 INFO [Listener at localhost/46647] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/java.io.tmpdir/Jetty_localhost_37143_datanode____.kds499/webapp 2023-07-18 03:55:33,437 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x891129be50ec140c: Processing first storage report for DS-70babad7-9ea0-4927-991b-f59f632e2026 from datanode 9fd06340-54ee-4f42-95a8-f0f5c6d6fc6d 2023-07-18 03:55:33,438 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x891129be50ec140c: from storage DS-70babad7-9ea0-4927-991b-f59f632e2026 node DatanodeRegistration(127.0.0.1:44219, datanodeUuid=9fd06340-54ee-4f42-95a8-f0f5c6d6fc6d, infoPort=45409, infoSecurePort=0, ipcPort=46647, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:33,438 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x891129be50ec140c: Processing first storage report for DS-83b59952-db9f-4ca5-b71c-0025ac416806 from datanode 9fd06340-54ee-4f42-95a8-f0f5c6d6fc6d 2023-07-18 03:55:33,438 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x891129be50ec140c: from storage DS-83b59952-db9f-4ca5-b71c-0025ac416806 node DatanodeRegistration(127.0.0.1:44219, datanodeUuid=9fd06340-54ee-4f42-95a8-f0f5c6d6fc6d, infoPort=45409, infoSecurePort=0, ipcPort=46647, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:33,492 INFO [Listener at localhost/46647] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37143 2023-07-18 03:55:33,510 WARN [Listener at localhost/46383] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 03:55:33,649 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf6c2a69ec29196fd: Processing first storage report for DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7 from datanode 1d376b4a-6efa-4b84-93e4-5173e152d9c5 2023-07-18 03:55:33,649 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf6c2a69ec29196fd: from storage DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7 node DatanodeRegistration(127.0.0.1:34991, datanodeUuid=1d376b4a-6efa-4b84-93e4-5173e152d9c5, infoPort=34177, infoSecurePort=0, ipcPort=46383, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:33,650 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf6c2a69ec29196fd: Processing first storage report for DS-79db125f-a7aa-4a80-9050-3cffe15efc0f from datanode 1d376b4a-6efa-4b84-93e4-5173e152d9c5 2023-07-18 03:55:33,650 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf6c2a69ec29196fd: from storage DS-79db125f-a7aa-4a80-9050-3cffe15efc0f node DatanodeRegistration(127.0.0.1:34991, datanodeUuid=1d376b4a-6efa-4b84-93e4-5173e152d9c5, infoPort=34177, infoSecurePort=0, ipcPort=46383, storageInfo=lv=-57;cid=testClusterID;nsid=705070083;c=1689652532356), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 03:55:33,729 DEBUG [Listener at localhost/46383] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399 2023-07-18 03:55:33,729 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,730 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,762 INFO [Listener at localhost/46383] util.FSUtils(471): Created version file at hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb with version=8 2023-07-18 03:55:33,763 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/hbase-staging 2023-07-18 03:55:33,765 INFO [Listener at localhost/46383] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:33,765 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,765 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,765 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:33,765 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,766 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:33,766 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:33,767 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33247 2023-07-18 03:55:33,768 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,769 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,771 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33247 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:33,774 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:332470x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:33,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33247-0x1017691f55f0007 connected 2023-07-18 03:55:33,789 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:33,789 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:33,790 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:33,793 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33247 2023-07-18 03:55:33,794 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33247 2023-07-18 03:55:33,794 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33247 2023-07-18 03:55:33,794 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33247 2023-07-18 03:55:33,794 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33247 2023-07-18 03:55:33,794 INFO [Listener at localhost/46383] master.HMaster(444): hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb, hbase.cluster.distributed=false 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:33,809 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:33,811 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45597 2023-07-18 03:55:33,812 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:33,813 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:33,814 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,815 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,817 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45597 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:33,822 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:455970x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:33,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45597-0x1017691f55f0008 connected 2023-07-18 03:55:33,824 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:33,824 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:33,825 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:33,825 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45597 2023-07-18 03:55:33,826 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45597 2023-07-18 03:55:33,826 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45597 2023-07-18 03:55:33,826 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45597 2023-07-18 03:55:33,826 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45597 2023-07-18 03:55:33,838 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:33,838 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,839 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,839 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:33,839 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,839 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:33,839 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:33,841 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36323 2023-07-18 03:55:33,841 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:33,842 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:33,843 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,845 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,845 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36323 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:33,849 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:363230x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:33,850 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:363230x0, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:33,850 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36323-0x1017691f55f0009 connected 2023-07-18 03:55:33,851 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:33,851 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:33,852 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36323 2023-07-18 03:55:33,852 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36323 2023-07-18 03:55:33,852 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36323 2023-07-18 03:55:33,853 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36323 2023-07-18 03:55:33,853 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36323 2023-07-18 03:55:33,864 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:33,864 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,864 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,865 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:33,865 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,865 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:33,865 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:33,867 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39019 2023-07-18 03:55:33,867 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:33,870 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:33,870 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,871 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,872 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39019 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:33,876 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:390190x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:33,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39019-0x1017691f55f000a connected 2023-07-18 03:55:33,877 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:33,878 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:33,878 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:33,879 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39019 2023-07-18 03:55:33,879 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39019 2023-07-18 03:55:33,879 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39019 2023-07-18 03:55:33,880 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39019 2023-07-18 03:55:33,880 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39019 2023-07-18 03:55:33,891 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:33,892 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,892 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,892 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:33,892 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:33,892 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:33,892 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:33,897 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33473 2023-07-18 03:55:33,897 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:33,898 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:33,899 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,900 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,901 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33473 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:33,906 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:334730x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:33,907 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:334730x0, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:33,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33473-0x1017691f55f000b connected 2023-07-18 03:55:33,908 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:33,908 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:33,909 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33473 2023-07-18 03:55:33,910 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33473 2023-07-18 03:55:33,910 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33473 2023-07-18 03:55:33,910 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33473 2023-07-18 03:55:33,911 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33473 2023-07-18 03:55:33,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:33,914 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 03:55:33,915 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:33,917 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:33,917 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:33,917 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:33,917 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:33,919 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:33,917 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:33,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 03:55:33,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,33247,1689652533764 from backup master directory 2023-07-18 03:55:33,924 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:33,924 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 03:55:33,924 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:33,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 03:55:33,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:33,956 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/hbase.id with ID: 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:33,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:33,979 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:33,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x70f6554f to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:34,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e7901ba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:34,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:34,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 03:55:34,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:34,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store-tmp 2023-07-18 03:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:55:34,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:34,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:34,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:34,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:34,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33247%2C1689652533764, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/oldWALs, maxLogs=10 2023-07-18 03:55:34,056 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:34,056 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:55:34,057 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:34,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 2023-07-18 03:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK], DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK]] 2023-07-18 03:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:34,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:34,065 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:34,067 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 03:55:34,068 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 03:55:34,069 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:34,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:34,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:34,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:34,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72206195, jitterRate=0.07595615088939667}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:55:34,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:34,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 03:55:34,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 03:55:34,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 03:55:34,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 03:55:34,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 03:55:34,083 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-18 03:55:34,083 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 03:55:34,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-18 03:55:34,085 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-18 03:55:34,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 03:55:34,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 03:55:34,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-18 03:55:34,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 03:55:34,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-18 03:55:34,099 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:34,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-18 03:55:34,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-18 03:55:34,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-18 03:55:34,103 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:34,103 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:34,103 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:34,103 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:34,103 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:34,103 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:34,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33247,1689652533764, sessionid=0x1017691f55f0007, setting cluster-up flag (Was=false) 2023-07-18 03:55:34,110 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:34,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-18 03:55:34,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:34,122 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:34,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-18 03:55:34,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:34,132 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.hbase-snapshot/.tmp 2023-07-18 03:55:34,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:34,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:34,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689652564148 2023-07-18 03:55:34,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 03:55:34,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 03:55:34,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 03:55:34,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 03:55:34,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 03:55:34,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 03:55:34,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 03:55:34,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 03:55:34,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 03:55:34,151 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:34,152 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-18 03:55:34,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 03:55:34,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 03:55:34,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652534152,5,FailOnTimeoutGroup] 2023-07-18 03:55:34,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652534152,5,FailOnTimeoutGroup] 2023-07-18 03:55:34,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 03:55:34,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,154 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:34,206 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:34,207 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:34,207 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:55:34,213 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(951): ClusterId : 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:34,215 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(951): ClusterId : 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:34,213 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(951): ClusterId : 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:34,226 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:34,217 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(951): ClusterId : 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:34,217 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:34,228 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:34,230 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:34,233 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:34,233 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:34,233 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:34,233 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:34,233 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:34,233 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:34,233 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:34,235 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:34,238 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:34,241 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:34,241 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:34,243 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:34,243 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ReadOnlyZKClient(139): Connect 0x228710aa to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:34,243 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ReadOnlyZKClient(139): Connect 0x3c7d2585 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:34,245 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ReadOnlyZKClient(139): Connect 0x18a1f13d to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:34,245 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ReadOnlyZKClient(139): Connect 0x28f10c33 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:34,267 DEBUG [RS:2;jenkins-hbase4:39019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c9e66f1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:34,268 DEBUG [RS:2;jenkins-hbase4:39019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@599cfb06, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:34,267 DEBUG [RS:3;jenkins-hbase4:33473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c40fa8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:34,268 DEBUG [RS:3;jenkins-hbase4:33473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@287f7bda, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:34,269 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:34,270 DEBUG [RS:0;jenkins-hbase4:45597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69cc0608, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:34,270 DEBUG [RS:1;jenkins-hbase4:36323] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36854ef5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:34,270 DEBUG [RS:0;jenkins-hbase4:45597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78dfc77a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:34,270 DEBUG [RS:1;jenkins-hbase4:36323] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7387c641, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:34,278 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:33473 2023-07-18 03:55:34,279 INFO [RS:3;jenkins-hbase4:33473] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:34,279 INFO [RS:3;jenkins-hbase4:33473] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:34,279 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:34,279 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45597 2023-07-18 03:55:34,279 INFO [RS:0;jenkins-hbase4:45597] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:34,279 INFO [RS:0;jenkins-hbase4:45597] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:34,279 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:34,279 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33247,1689652533764 with isa=jenkins-hbase4.apache.org/172.31.14.131:33473, startcode=1689652533891 2023-07-18 03:55:34,280 DEBUG [RS:3;jenkins-hbase4:33473] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:34,280 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33247,1689652533764 with isa=jenkins-hbase4.apache.org/172.31.14.131:45597, startcode=1689652533808 2023-07-18 03:55:34,280 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:39019 2023-07-18 03:55:34,280 DEBUG [RS:0;jenkins-hbase4:45597] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:34,280 INFO [RS:2;jenkins-hbase4:39019] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:34,280 INFO [RS:2;jenkins-hbase4:39019] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:34,280 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:34,281 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:55:34,281 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33247,1689652533764 with isa=jenkins-hbase4.apache.org/172.31.14.131:39019, startcode=1689652533864 2023-07-18 03:55:34,281 DEBUG [RS:2;jenkins-hbase4:39019] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:34,282 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:36323 2023-07-18 03:55:34,282 INFO [RS:1;jenkins-hbase4:36323] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:34,282 INFO [RS:1;jenkins-hbase4:36323] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:34,282 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:34,285 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33247,1689652533764 with isa=jenkins-hbase4.apache.org/172.31.14.131:36323, startcode=1689652533838 2023-07-18 03:55:34,285 DEBUG [RS:1;jenkins-hbase4:36323] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:34,285 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38731, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:34,286 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40763, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:34,292 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,292 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:55:34,293 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,293 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:55:34,295 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:55:34,295 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36001 2023-07-18 03:55:34,295 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:34,295 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59189, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:34,296 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:55:34,297 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:55:34,297 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36001 2023-07-18 03:55:34,297 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:34,298 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:55:34,298 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36001 2023-07-18 03:55:34,298 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:34,298 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55733, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:34,299 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,301 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:34,301 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:55:34,301 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36001 2023-07-18 03:55:34,301 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:34,302 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:34,302 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ZKUtil(162): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,303 WARN [RS:3;jenkins-hbase4:33473] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:34,303 INFO [RS:3;jenkins-hbase4:33473] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:34,303 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,303 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:55:34,304 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ZKUtil(162): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,304 WARN [RS:1;jenkins-hbase4:36323] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:34,304 INFO [RS:1;jenkins-hbase4:36323] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:34,304 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,312 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:55:34,307 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ZKUtil(162): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,307 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ZKUtil(162): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,315 WARN [RS:2;jenkins-hbase4:39019] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:34,315 INFO [RS:2;jenkins-hbase4:39019] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:34,315 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,304 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,315 WARN [RS:0;jenkins-hbase4:45597] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:34,316 INFO [RS:0;jenkins-hbase4:45597] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:34,316 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,317 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:55:34,318 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:55:34,319 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,320 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:55:34,330 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39019,1689652533864] 2023-07-18 03:55:34,330 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36323,1689652533838] 2023-07-18 03:55:34,330 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45597,1689652533808] 2023-07-18 03:55:34,330 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33473,1689652533891] 2023-07-18 03:55:34,330 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:55:34,334 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:55:34,341 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:55:34,346 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ZKUtil(162): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,346 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ZKUtil(162): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,346 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ZKUtil(162): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,347 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ZKUtil(162): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,348 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ZKUtil(162): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,348 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ZKUtil(162): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,348 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ZKUtil(162): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,349 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:34,349 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ZKUtil(162): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,349 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:34,350 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60852402, jitterRate=-0.0932285487651825}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:55:34,350 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:55:34,350 INFO [RS:2;jenkins-hbase4:39019] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:34,350 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:55:34,350 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:55:34,350 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:55:34,350 DEBUG [RS:1;jenkins-hbase4:36323] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:34,350 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:55:34,350 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:55:34,350 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ZKUtil(162): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,351 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:55:34,352 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:55:34,353 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ZKUtil(162): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,353 INFO [RS:1;jenkins-hbase4:36323] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:34,354 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ZKUtil(162): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,354 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:34,354 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-18 03:55:34,354 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ZKUtil(162): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 03:55:34,355 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ZKUtil(162): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,356 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ZKUtil(162): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,356 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ZKUtil(162): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,356 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ZKUtil(162): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,358 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:34,358 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:34,358 INFO [RS:3;jenkins-hbase4:33473] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:34,358 INFO [RS:0;jenkins-hbase4:45597] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:34,359 INFO [RS:2;jenkins-hbase4:39019] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:34,363 INFO [RS:2;jenkins-hbase4:39019] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:34,363 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,367 INFO [RS:1;jenkins-hbase4:36323] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:34,373 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 03:55:34,374 INFO [RS:3;jenkins-hbase4:33473] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:34,391 INFO [RS:3;jenkins-hbase4:33473] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:34,391 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,394 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:34,395 INFO [RS:0;jenkins-hbase4:45597] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:34,398 INFO [RS:1;jenkins-hbase4:36323] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:34,398 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,398 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:34,409 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-18 03:55:34,415 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:34,430 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,430 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,430 INFO [RS:0;jenkins-hbase4:45597] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:34,430 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,430 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,430 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,430 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,430 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:3;jenkins-hbase4:33473] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,431 DEBUG [RS:2;jenkins-hbase4:39019] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,430 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,431 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,432 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:34,435 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,435 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,435 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,435 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,435 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,435 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,435 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,436 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,436 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,436 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:34,436 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,436 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,437 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:34,437 DEBUG [RS:1;jenkins-hbase4:36323] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,437 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,437 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,437 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,437 DEBUG [RS:0;jenkins-hbase4:45597] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:34,437 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,439 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,439 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,439 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,439 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,439 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,450 INFO [RS:3;jenkins-hbase4:33473] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:34,450 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33473,1689652533891-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,451 INFO [RS:2;jenkins-hbase4:39019] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:34,451 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39019,1689652533864-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,453 INFO [RS:1;jenkins-hbase4:36323] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:34,453 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36323,1689652533838-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,459 INFO [RS:0;jenkins-hbase4:45597] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:34,459 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45597,1689652533808-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,465 INFO [RS:3;jenkins-hbase4:33473] regionserver.Replication(203): jenkins-hbase4.apache.org,33473,1689652533891 started 2023-07-18 03:55:34,465 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33473,1689652533891, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33473, sessionid=0x1017691f55f000b 2023-07-18 03:55:34,465 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:34,465 DEBUG [RS:3;jenkins-hbase4:33473] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,466 DEBUG [RS:3;jenkins-hbase4:33473] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33473,1689652533891' 2023-07-18 03:55:34,466 DEBUG [RS:3;jenkins-hbase4:33473] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 03:55:34,466 INFO [RS:2;jenkins-hbase4:39019] regionserver.Replication(203): jenkins-hbase4.apache.org,39019,1689652533864 started 2023-07-18 03:55:34,466 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39019,1689652533864, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39019, sessionid=0x1017691f55f000a 2023-07-18 03:55:34,466 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:34,466 DEBUG [RS:3;jenkins-hbase4:33473] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 03:55:34,466 DEBUG [RS:2;jenkins-hbase4:39019] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,466 DEBUG [RS:2;jenkins-hbase4:39019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39019,1689652533864' 2023-07-18 03:55:34,466 DEBUG [RS:2;jenkins-hbase4:39019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 03:55:34,468 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:34,468 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:34,468 DEBUG [RS:3;jenkins-hbase4:33473] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:34,468 DEBUG [RS:3;jenkins-hbase4:33473] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33473,1689652533891' 2023-07-18 03:55:34,468 DEBUG [RS:3;jenkins-hbase4:33473] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 03:55:34,469 INFO [RS:1;jenkins-hbase4:36323] regionserver.Replication(203): jenkins-hbase4.apache.org,36323,1689652533838 started 2023-07-18 03:55:34,469 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36323,1689652533838, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36323, sessionid=0x1017691f55f0009 2023-07-18 03:55:34,469 DEBUG [RS:3;jenkins-hbase4:33473] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 03:55:34,469 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:34,469 DEBUG [RS:1;jenkins-hbase4:36323] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,469 DEBUG [RS:1;jenkins-hbase4:36323] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36323,1689652533838' 2023-07-18 03:55:34,469 DEBUG [RS:1;jenkins-hbase4:36323] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 03:55:34,471 DEBUG [RS:2;jenkins-hbase4:39019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 03:55:34,471 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:34,472 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:34,472 DEBUG [RS:2;jenkins-hbase4:39019] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,472 DEBUG [RS:2;jenkins-hbase4:39019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39019,1689652533864' 2023-07-18 03:55:34,472 DEBUG [RS:2;jenkins-hbase4:39019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 03:55:34,472 DEBUG [RS:2;jenkins-hbase4:39019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 03:55:34,473 DEBUG [RS:2;jenkins-hbase4:39019] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:34,473 INFO [RS:2;jenkins-hbase4:39019] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:34,473 INFO [RS:2;jenkins-hbase4:39019] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:34,475 DEBUG [RS:3;jenkins-hbase4:33473] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:34,475 DEBUG [RS:1;jenkins-hbase4:36323] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 03:55:34,475 INFO [RS:3;jenkins-hbase4:33473] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:34,475 INFO [RS:3;jenkins-hbase4:33473] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:34,476 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:34,476 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:34,476 DEBUG [RS:1;jenkins-hbase4:36323] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:34,476 DEBUG [RS:1;jenkins-hbase4:36323] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36323,1689652533838' 2023-07-18 03:55:34,476 DEBUG [RS:1;jenkins-hbase4:36323] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 03:55:34,476 DEBUG [RS:1;jenkins-hbase4:36323] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 03:55:34,477 DEBUG [RS:1;jenkins-hbase4:36323] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:34,477 INFO [RS:1;jenkins-hbase4:36323] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:34,477 INFO [RS:1;jenkins-hbase4:36323] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:34,479 INFO [RS:0;jenkins-hbase4:45597] regionserver.Replication(203): jenkins-hbase4.apache.org,45597,1689652533808 started 2023-07-18 03:55:34,479 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45597,1689652533808, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45597, sessionid=0x1017691f55f0008 2023-07-18 03:55:34,479 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:34,479 DEBUG [RS:0;jenkins-hbase4:45597] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,479 DEBUG [RS:0;jenkins-hbase4:45597] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45597,1689652533808' 2023-07-18 03:55:34,479 DEBUG [RS:0;jenkins-hbase4:45597] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 03:55:34,480 DEBUG [RS:0;jenkins-hbase4:45597] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 03:55:34,480 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:34,480 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:34,480 DEBUG [RS:0;jenkins-hbase4:45597] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:34,480 DEBUG [RS:0;jenkins-hbase4:45597] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45597,1689652533808' 2023-07-18 03:55:34,480 DEBUG [RS:0;jenkins-hbase4:45597] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 03:55:34,481 DEBUG [RS:0;jenkins-hbase4:45597] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 03:55:34,482 DEBUG [RS:0;jenkins-hbase4:45597] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:34,482 INFO [RS:0;jenkins-hbase4:45597] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:34,482 INFO [RS:0;jenkins-hbase4:45597] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:34,576 INFO [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39019%2C1689652533864, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:34,578 INFO [RS:3;jenkins-hbase4:33473] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33473%2C1689652533891, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:34,579 INFO [RS:1;jenkins-hbase4:36323] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36323%2C1689652533838, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:34,580 DEBUG [jenkins-hbase4:33247] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-18 03:55:34,580 DEBUG [jenkins-hbase4:33247] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:34,583 DEBUG [jenkins-hbase4:33247] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:34,583 DEBUG [jenkins-hbase4:33247] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:34,583 DEBUG [jenkins-hbase4:33247] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 03:55:34,583 DEBUG [jenkins-hbase4:33247] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 03:55:34,583 DEBUG [jenkins-hbase4:33247] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:34,585 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39019,1689652533864, state=OPENING 2023-07-18 03:55:34,586 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-18 03:55:34,587 INFO [RS:0;jenkins-hbase4:45597] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45597%2C1689652533808, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:34,588 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:34,589 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39019,1689652533864}] 2023-07-18 03:55:34,589 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 03:55:34,627 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:55:34,627 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:34,627 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:55:34,630 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:34,630 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:55:34,630 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:34,630 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:34,631 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:55:34,631 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:34,644 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:34,645 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:55:34,647 INFO [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864/jenkins-hbase4.apache.org%2C39019%2C1689652533864.1689652534578 2023-07-18 03:55:34,654 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:34,658 DEBUG [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK], DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK]] 2023-07-18 03:55:34,661 INFO [RS:1;jenkins-hbase4:36323] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838/jenkins-hbase4.apache.org%2C36323%2C1689652533838.1689652534580 2023-07-18 03:55:34,663 DEBUG [RS:1;jenkins-hbase4:36323] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK], DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK]] 2023-07-18 03:55:34,663 INFO [RS:3;jenkins-hbase4:33473] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891/jenkins-hbase4.apache.org%2C33473%2C1689652533891.1689652534579 2023-07-18 03:55:34,664 DEBUG [RS:3;jenkins-hbase4:33473] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK]] 2023-07-18 03:55:34,667 INFO [RS:0;jenkins-hbase4:45597] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808/jenkins-hbase4.apache.org%2C45597%2C1689652533808.1689652534588 2023-07-18 03:55:34,668 DEBUG [RS:0;jenkins-hbase4:45597] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK], DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK]] 2023-07-18 03:55:34,759 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:34,759 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:34,762 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43274, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:34,772 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 03:55:34,772 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:34,774 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39019%2C1689652533864.meta, suffix=.meta, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:34,794 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:34,795 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:55:34,795 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:34,805 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864/jenkins-hbase4.apache.org%2C39019%2C1689652533864.meta.1689652534775.meta 2023-07-18 03:55:34,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK], DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK]] 2023-07-18 03:55:34,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:34,806 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 03:55:34,806 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 03:55:34,807 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 03:55:34,807 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 03:55:34,807 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:34,807 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 03:55:34,807 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 03:55:34,809 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:55:34,811 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:55:34,811 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:55:34,811 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:55:34,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:55:34,814 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:34,814 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:34,814 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:55:34,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:55:34,817 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:55:34,817 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:55:34,817 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:55:34,818 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:34,820 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:55:34,822 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:55:34,826 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:55:34,828 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:55:34,829 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71416717, jitterRate=0.06419201195240021}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:55:34,829 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:55:34,830 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689652534759 2023-07-18 03:55:34,836 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 03:55:34,837 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 03:55:34,837 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39019,1689652533864, state=OPEN 2023-07-18 03:55:34,841 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 03:55:34,841 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 03:55:34,844 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-18 03:55:34,844 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39019,1689652533864 in 253 msec 2023-07-18 03:55:34,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-18 03:55:34,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 490 msec 2023-07-18 03:55:34,851 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 715 msec 2023-07-18 03:55:34,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689652534851, completionTime=-1 2023-07-18 03:55:34,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-18 03:55:34,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 03:55:34,857 DEBUG [hconnection-0x5ceee78-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:34,859 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43282, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:34,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-18 03:55:34,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689652594861 2023-07-18 03:55:34,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689652654861 2023-07-18 03:55:34,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 9 msec 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33247,1689652533764-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33247,1689652533764-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33247,1689652533764-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33247, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-18 03:55:34,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:34,871 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-18 03:55:34,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-18 03:55:34,873 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 03:55:34,874 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 03:55:34,880 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:34,880 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 empty. 2023-07-18 03:55:34,881 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:34,881 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-18 03:55:34,961 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:34,962 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6596a12435c5c3b7add032ab29aea733, NAME => 'hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp 2023-07-18 03:55:34,985 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:34,985 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 6596a12435c5c3b7add032ab29aea733, disabling compactions & flushes 2023-07-18 03:55:34,985 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:34,985 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:34,985 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. after waiting 0 ms 2023-07-18 03:55:34,985 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:34,985 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:34,985 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:55:34,989 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 03:55:34,993 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652534993"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689652534993"}]},"ts":"1689652534993"} 2023-07-18 03:55:34,997 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 03:55:34,998 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 03:55:34,999 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652534999"}]},"ts":"1689652534999"} 2023-07-18 03:55:35,001 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-18 03:55:35,006 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:35,007 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:35,007 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:35,007 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 03:55:35,007 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 03:55:35,007 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:35,007 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN}] 2023-07-18 03:55:35,010 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN 2023-07-18 03:55:35,011 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45597,1689652533808; forceNewPlan=false, retain=false 2023-07-18 03:55:35,162 INFO [jenkins-hbase4:33247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 03:55:35,164 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6596a12435c5c3b7add032ab29aea733, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:35,164 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652535163"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652535163"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652535163"}]},"ts":"1689652535163"} 2023-07-18 03:55:35,167 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6596a12435c5c3b7add032ab29aea733, server=jenkins-hbase4.apache.org,45597,1689652533808}] 2023-07-18 03:55:35,322 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:35,322 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:35,324 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39446, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:35,334 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:35,336 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6596a12435c5c3b7add032ab29aea733, NAME => 'hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:35,336 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,336 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:35,336 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,336 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,345 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,348 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info 2023-07-18 03:55:35,348 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info 2023-07-18 03:55:35,350 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6596a12435c5c3b7add032ab29aea733 columnFamilyName info 2023-07-18 03:55:35,351 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(310): Store=6596a12435c5c3b7add032ab29aea733/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:35,354 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,355 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,360 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:35,363 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:35,364 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6596a12435c5c3b7add032ab29aea733; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61085159, jitterRate=-0.0897601991891861}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:55:35,364 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:55:35,366 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., pid=6, masterSystemTime=1689652535322 2023-07-18 03:55:35,371 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:35,372 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:35,373 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6596a12435c5c3b7add032ab29aea733, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:35,373 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652535373"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652535373"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652535373"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652535373"}]},"ts":"1689652535373"} 2023-07-18 03:55:35,380 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-18 03:55:35,380 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6596a12435c5c3b7add032ab29aea733, server=jenkins-hbase4.apache.org,45597,1689652533808 in 209 msec 2023-07-18 03:55:35,384 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-18 03:55:35,385 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN in 373 msec 2023-07-18 03:55:35,386 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 03:55:35,386 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652535386"}]},"ts":"1689652535386"} 2023-07-18 03:55:35,388 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-18 03:55:35,391 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 03:55:35,394 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 522 msec 2023-07-18 03:55:35,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-18 03:55:35,478 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-18 03:55:35,478 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:35,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:35,484 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39458, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:35,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-18 03:55:35,501 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-18 03:55:35,506 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 18 msec 2023-07-18 03:55:35,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-18 03:55:35,519 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-18 03:55:35,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-07-18 03:55:35,534 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-18 03:55:35,537 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-18 03:55:35,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.612sec 2023-07-18 03:55:35,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 03:55:35,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 03:55:35,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 03:55:35,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33247,1689652533764-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 03:55:35,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33247,1689652533764-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 03:55:35,542 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 03:55:35,617 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x4be4c0d5 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:35,629 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@409f3945, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:35,632 DEBUG [hconnection-0x4562c9d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:35,634 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43288, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:35,641 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:35,658 DEBUG [Listener at localhost/46383] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-18 03:55:35,665 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-18 03:55:35,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:35,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-18 03:55:35,692 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 03:55:35,695 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 03:55:35,698 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:35,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-18 03:55:35,700 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 empty. 2023-07-18 03:55:35,700 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:35,700 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-18 03:55:35,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:35,738 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:35,739 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => f1f3df17ddf51de240de231140850f62, NAME => 't1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp 2023-07-18 03:55:35,757 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:35,757 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing f1f3df17ddf51de240de231140850f62, disabling compactions & flushes 2023-07-18 03:55:35,757 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:35,757 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:35,757 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. after waiting 0 ms 2023-07-18 03:55:35,757 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:35,757 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:35,757 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for f1f3df17ddf51de240de231140850f62: 2023-07-18 03:55:35,762 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 03:55:35,764 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652535764"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689652535764"}]},"ts":"1689652535764"} 2023-07-18 03:55:35,767 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 03:55:35,768 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 03:55:35,769 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652535768"}]},"ts":"1689652535768"} 2023-07-18 03:55:35,771 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-18 03:55:35,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:35,779 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:35,779 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:35,779 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:35,779 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN}] 2023-07-18 03:55:35,783 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN 2023-07-18 03:55:35,784 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36729,1689652529501; forceNewPlan=false, retain=false 2023-07-18 03:55:35,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:35,869 INFO [jenkins-hbase4:42337Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:35,869 INFO [jenkins-hbase4:36729Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:35,934 INFO [jenkins-hbase4:41639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 03:55:35,939 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f1f3df17ddf51de240de231140850f62, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:35,939 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652535937"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652535937"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652535937"}]},"ts":"1689652535937"} 2023-07-18 03:55:35,942 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f1f3df17ddf51de240de231140850f62, server=jenkins-hbase4.apache.org,36729,1689652529501}] 2023-07-18 03:55:36,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:36,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:36,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1f3df17ddf51de240de231140850f62, NAME => 't1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:36,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:36,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,105 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,107 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1 2023-07-18 03:55:36,107 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1 2023-07-18 03:55:36,107 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1f3df17ddf51de240de231140850f62 columnFamilyName cf1 2023-07-18 03:55:36,108 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(310): Store=f1f3df17ddf51de240de231140850f62/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:36,108 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,110 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep 2023-07-18 03:55:36,110 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep 2023-07-18 03:55:36,110 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1f3df17ddf51de240de231140850f62 columnFamilyName norep 2023-07-18 03:55:36,111 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(310): Store=f1f3df17ddf51de240de231140850f62/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:36,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:55:36,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:36,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:36,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1f3df17ddf51de240de231140850f62; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60072084, jitterRate=-0.10485619306564331}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:55:36,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1f3df17ddf51de240de231140850f62: 2023-07-18 03:55:36,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62., pid=11, masterSystemTime=1689652536095 2023-07-18 03:55:36,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:36,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:36,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f1f3df17ddf51de240de231140850f62, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:36,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652536125"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652536125"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652536125"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652536125"}]},"ts":"1689652536125"} 2023-07-18 03:55:36,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-18 03:55:36,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f1f3df17ddf51de240de231140850f62, server=jenkins-hbase4.apache.org,36729,1689652529501 in 186 msec 2023-07-18 03:55:36,133 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-18 03:55:36,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN in 351 msec 2023-07-18 03:55:36,135 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 03:55:36,135 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652536135"}]},"ts":"1689652536135"} 2023-07-18 03:55:36,136 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-18 03:55:36,139 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 03:55:36,141 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 455 msec 2023-07-18 03:55:36,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:36,333 INFO [Listener at localhost/46383] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-18 03:55:36,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:36,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-18 03:55:36,339 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 03:55:36,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-18 03:55:36,341 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 03:55:36,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:36,344 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,346 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 empty. 2023-07-18 03:55:36,348 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,348 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-18 03:55:36,373 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:36,375 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => f95930cf45f370dfce61fdb9733fd578, NAME => 't2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.tmp 2023-07-18 03:55:36,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:36,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing f95930cf45f370dfce61fdb9733fd578, disabling compactions & flushes 2023-07-18 03:55:36,394 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. after waiting 0 ms 2023-07-18 03:55:36,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,394 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,394 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for f95930cf45f370dfce61fdb9733fd578: 2023-07-18 03:55:36,400 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 03:55:36,402 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652536402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689652536402"}]},"ts":"1689652536402"} 2023-07-18 03:55:36,404 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 03:55:36,405 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 03:55:36,405 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652536405"}]},"ts":"1689652536405"} 2023-07-18 03:55:36,407 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-18 03:55:36,411 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:36,412 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:36,412 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:36,412 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:36,413 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN}] 2023-07-18 03:55:36,415 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN 2023-07-18 03:55:36,416 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42337,1689652529544; forceNewPlan=false, retain=false 2023-07-18 03:55:36,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:36,567 INFO [jenkins-hbase4:41639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 03:55:36,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f95930cf45f370dfce61fdb9733fd578, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:36,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652536568"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652536568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652536568"}]},"ts":"1689652536568"} 2023-07-18 03:55:36,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure f95930cf45f370dfce61fdb9733fd578, server=jenkins-hbase4.apache.org,42337,1689652529544}] 2023-07-18 03:55:36,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:36,725 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:36,726 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:36,728 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41612, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:36,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f95930cf45f370dfce61fdb9733fd578, NAME => 't2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:36,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:36,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,737 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,740 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1 2023-07-18 03:55:36,740 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1 2023-07-18 03:55:36,741 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f95930cf45f370dfce61fdb9733fd578 columnFamilyName cf1 2023-07-18 03:55:36,741 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(310): Store=f95930cf45f370dfce61fdb9733fd578/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:36,741 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,743 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep 2023-07-18 03:55:36,743 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep 2023-07-18 03:55:36,743 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f95930cf45f370dfce61fdb9733fd578 columnFamilyName norep 2023-07-18 03:55:36,744 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(310): Store=f95930cf45f370dfce61fdb9733fd578/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:36,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:55:36,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:36,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:36,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f95930cf45f370dfce61fdb9733fd578; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64184268, jitterRate=-0.04357987642288208}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:55:36,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f95930cf45f370dfce61fdb9733fd578: 2023-07-18 03:55:36,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578., pid=14, masterSystemTime=1689652536725 2023-07-18 03:55:36,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:36,767 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f95930cf45f370dfce61fdb9733fd578, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:36,767 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652536766"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652536766"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652536766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652536766"}]},"ts":"1689652536766"} 2023-07-18 03:55:36,770 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:55:36,773 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-18 03:55:36,774 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure f95930cf45f370dfce61fdb9733fd578, server=jenkins-hbase4.apache.org,42337,1689652529544 in 199 msec 2023-07-18 03:55:36,777 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-18 03:55:36,778 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN in 362 msec 2023-07-18 03:55:36,780 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 03:55:36,780 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652536780"}]},"ts":"1689652536780"} 2023-07-18 03:55:36,785 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-18 03:55:36,799 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 03:55:36,806 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 464 msec 2023-07-18 03:55:36,938 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-18 03:55:36,940 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-18 03:55:36,941 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-18 03:55:36,941 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-18 03:55:36,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:36,947 INFO [Listener at localhost/46383] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-18 03:55:36,951 DEBUG [Listener at localhost/46383] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-18 03:55:36,953 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40378, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-18 03:55:36,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:36,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-18 03:55:36,962 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 03:55:36,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-18 03:55:36,965 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 03:55:36,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:36,968 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:55:36,969 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 empty. 2023-07-18 03:55:36,969 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:55:36,969 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-18 03:55:37,001 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:37,003 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6d104435467e1000121c02223fd064d0, NAME => 't1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp 2023-07-18 03:55:37,029 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:37,029 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 6d104435467e1000121c02223fd064d0, disabling compactions & flushes 2023-07-18 03:55:37,029 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,029 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,029 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. after waiting 0 ms 2023-07-18 03:55:37,029 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,029 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,029 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:55:37,032 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 03:55:37,034 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652537034"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689652537034"}]},"ts":"1689652537034"} 2023-07-18 03:55:37,037 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 03:55:37,038 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 03:55:37,038 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652537038"}]},"ts":"1689652537038"} 2023-07-18 03:55:37,040 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-18 03:55:37,045 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:37,046 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:37,046 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:37,046 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 03:55:37,046 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 03:55:37,046 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:37,047 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN}] 2023-07-18 03:55:37,049 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN 2023-07-18 03:55:37,050 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39019,1689652533864; forceNewPlan=false, retain=false 2023-07-18 03:55:37,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:37,200 INFO [jenkins-hbase4:33247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 03:55:37,202 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6d104435467e1000121c02223fd064d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:37,202 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652537202"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652537202"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652537202"}]},"ts":"1689652537202"} 2023-07-18 03:55:37,205 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 6d104435467e1000121c02223fd064d0, server=jenkins-hbase4.apache.org,39019,1689652533864}] 2023-07-18 03:55:37,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:37,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6d104435467e1000121c02223fd064d0, NAME => 't1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,365 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,366 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1 2023-07-18 03:55:37,366 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1 2023-07-18 03:55:37,367 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d104435467e1000121c02223fd064d0 columnFamilyName cf1 2023-07-18 03:55:37,367 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(310): Store=6d104435467e1000121c02223fd064d0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:37,368 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,369 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/norep 2023-07-18 03:55:37,369 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/norep 2023-07-18 03:55:37,369 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d104435467e1000121c02223fd064d0 columnFamilyName norep 2023-07-18 03:55:37,370 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(310): Store=6d104435467e1000121c02223fd064d0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:37,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:55:37,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:37,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:37,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6d104435467e1000121c02223fd064d0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62606036, jitterRate=-0.06709736585617065}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:55:37,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:55:37,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0., pid=11, masterSystemTime=1689652537357 2023-07-18 03:55:37,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:37,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6d104435467e1000121c02223fd064d0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:37,384 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652537384"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652537384"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652537384"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652537384"}]},"ts":"1689652537384"} 2023-07-18 03:55:37,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-18 03:55:37,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 6d104435467e1000121c02223fd064d0, server=jenkins-hbase4.apache.org,39019,1689652533864 in 181 msec 2023-07-18 03:55:37,392 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-18 03:55:37,393 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN in 343 msec 2023-07-18 03:55:37,393 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 03:55:37,394 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652537394"}]},"ts":"1689652537394"} 2023-07-18 03:55:37,396 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-18 03:55:37,402 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 03:55:37,404 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 445 msec 2023-07-18 03:55:37,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 03:55:37,572 INFO [Listener at localhost/46383] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-18 03:55:37,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:37,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-18 03:55:37,578 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 03:55:37,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-18 03:55:37,580 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 03:55:37,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:37,583 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,584 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 empty. 2023-07-18 03:55:37,584 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,584 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-18 03:55:37,610 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 03:55:37,611 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => d59bee2bde49e7e5abb27771a5bfd3f4, NAME => 't2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.tmp 2023-07-18 03:55:37,631 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:37,631 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing d59bee2bde49e7e5abb27771a5bfd3f4, disabling compactions & flushes 2023-07-18 03:55:37,631 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,631 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,631 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. after waiting 0 ms 2023-07-18 03:55:37,631 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,631 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,631 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:55:37,635 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 03:55:37,637 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652537637"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689652537637"}]},"ts":"1689652537637"} 2023-07-18 03:55:37,640 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 03:55:37,641 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 03:55:37,642 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652537641"}]},"ts":"1689652537641"} 2023-07-18 03:55:37,644 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-18 03:55:37,649 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 03:55:37,651 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 03:55:37,651 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 03:55:37,651 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 03:55:37,651 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 03:55:37,651 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 03:55:37,652 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN}] 2023-07-18 03:55:37,655 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN 2023-07-18 03:55:37,656 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33473,1689652533891; forceNewPlan=false, retain=false 2023-07-18 03:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:37,806 INFO [jenkins-hbase4:33247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 03:55:37,808 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:37,808 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652537808"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652537808"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652537808"}]},"ts":"1689652537808"} 2023-07-18 03:55:37,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure d59bee2bde49e7e5abb27771a5bfd3f4, server=jenkins-hbase4.apache.org,33473,1689652533891}] 2023-07-18 03:55:37,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:37,964 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:37,964 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:37,966 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33728, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:37,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d59bee2bde49e7e5abb27771a5bfd3f4, NAME => 't2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:37,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:37,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,975 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,977 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1 2023-07-18 03:55:37,977 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1 2023-07-18 03:55:37,977 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d59bee2bde49e7e5abb27771a5bfd3f4 columnFamilyName cf1 2023-07-18 03:55:37,978 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(310): Store=d59bee2bde49e7e5abb27771a5bfd3f4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:37,978 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,979 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/norep 2023-07-18 03:55:37,980 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/norep 2023-07-18 03:55:37,980 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d59bee2bde49e7e5abb27771a5bfd3f4 columnFamilyName norep 2023-07-18 03:55:37,980 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(310): Store=d59bee2bde49e7e5abb27771a5bfd3f4/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:37,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:55:37,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:37,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 03:55:37,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d59bee2bde49e7e5abb27771a5bfd3f4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73454787, jitterRate=0.0945616215467453}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:55:37,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:55:37,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4., pid=14, masterSystemTime=1689652537964 2023-07-18 03:55:37,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:37,996 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:37,996 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652537996"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652537996"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652537996"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652537996"}]},"ts":"1689652537996"} 2023-07-18 03:55:38,001 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-18 03:55:38,001 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure d59bee2bde49e7e5abb27771a5bfd3f4, server=jenkins-hbase4.apache.org,33473,1689652533891 in 188 msec 2023-07-18 03:55:38,003 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-18 03:55:38,003 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN in 350 msec 2023-07-18 03:55:38,004 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 03:55:38,004 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689652538004"}]},"ts":"1689652538004"} 2023-07-18 03:55:38,006 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-18 03:55:38,008 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 03:55:38,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 434 msec 2023-07-18 03:55:38,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 03:55:38,186 INFO [Listener at localhost/46383] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-18 03:55:38,187 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x7202ca53 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:38,195 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e2d32d5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:38,195 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x1ffdd212 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:38,205 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b49d3ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:38,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:60939:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-18 03:55:38,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-18 03:55:38,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 03:55:38,245 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 03:55:38,245 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@c7b17a3check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:38,249 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@c7b17a3check-peer-cluster-id0x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:38,250 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@c7b17a3check-peer-cluster-id-0x1017691f55f0014 connected 2023-07-18 03:55:38,303 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 03:55:38,304 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-18 03:55:38,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 03:55:38,352 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@c7b17a3check-peer-cluster-id-0x1017691f55f0014, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:38,362 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:38,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-18 03:55:38,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 03:55:38,539 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42337] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-18 03:55:38,539 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36729] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-18 03:55:38,541 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-18 03:55:38,541 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-18 03:55:38,560 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 03:55:38,560 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 03:55:38,585 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:55:38,585 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:55:38,617 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-18 03:55:38,617 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-18 03:55:38,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] master.HMaster(3641): Remote procedure done, pid=17 2023-07-18 03:55:38,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41639] master.HMaster(3641): Remote procedure done, pid=16 2023-07-18 03:55:38,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41639] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,42337,1689652529544 suceeded 2023-07-18 03:55:38,620 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41639] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,36729,1689652529501 suceeded 2023-07-18 03:55:38,626 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 251 msec 2023-07-18 03:55:38,627 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-18 03:55:38,627 INFO [PEWorker-2] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:60939:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-18 03:55:38,627 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 251 msec 2023-07-18 03:55:38,631 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 406 msec 2023-07-18 03:55:38,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1bafb0f2 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:38,643 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x620c749e to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:38,650 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@259b6ffb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:38,650 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6cdef1ef, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:38,654 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:38,654 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:38,659 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:38,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f0017 connected 2023-07-18 03:55:38,660 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:38,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f0018 connected 2023-07-18 03:55:38,661 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:38,662 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:38,664 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C36729%2C1689652529501 2023-07-18 03:55:38,664 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42337%2C1689652529544 2023-07-18 03:55:38,669 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 03:55:38,669 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 03:55:38,671 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C36729%2C1689652529501 2023-07-18 03:55:38,672 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42337%2C1689652529544 2023-07-18 03:55:38,752 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992, lastWalPosition=901, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 03:55:38,753 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-18 03:55:38,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 03:55:38,839 DEBUG [Listener at localhost/46383] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-18 03:55:38,841 DEBUG [hconnection-0x6b831c8f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:38,843 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48480, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:38,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 1107 2023-07-18 03:55:38,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-18 03:55:38,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 2 WAL entries eligible for replication 2023-07-18 03:55:38,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 1107 2023-07-18 03:55:38,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/4=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/5=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992, lastWalPosition=1107, nbRowKeys=2, nbHFiles=0, heapSize=416, lastSeqIds={}, endOfFile=false,usedBufferSize=416] 2023-07-18 03:55:38,880 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:38,883 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43294, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-18 03:55:38,889 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:38,890 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] zookeeper.ReadOnlyZKClient(139): Connect 0x59049db1 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:38,895 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@55d31a39, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:38,905 DEBUG [hconnection-0x8094b60-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:38,906 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43298, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:38,954 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:38,975 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-18 03:55:38,984 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 8336 2023-07-18 03:55:38,985 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 69 WAL entries eligible for replication 2023-07-18 03:55:38,985 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 8336 2023-07-18 03:55:38,985 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/6=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/7=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/8=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/9=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/10=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/11=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/12=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/13=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/14=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/15=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/16=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/17=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/18=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/19=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/20=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/21=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/22=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/23=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/24=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/25=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/26=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/27=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/28=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/29=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/30=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/31=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/32=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/33=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/34=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/35=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/36=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/37=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/38=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/39=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/40=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/41=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/42=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/43=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/44=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/45=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/46=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/47=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/48=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/49=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/50=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/51=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/52=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/53=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/54=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/55=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/56=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/57=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/58=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/59=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/60=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/61=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/62=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/63=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/64=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/65=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/66=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/67=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/68=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/69=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/70=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/71=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/72=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/73=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/74=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992, lastWalPosition=8336, nbRowKeys=69, nbHFiles=0, heapSize=14352, lastSeqIds={}, endOfFile=false,usedBufferSize=14352] 2023-07-18 03:55:38,992 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:39,001 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:39,017 DEBUG [Listener at localhost/46383] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,020 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41628, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,093 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 4984 2023-07-18 03:55:39,093 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceWALReader(162): Read 44 WAL entries eligible for replication 2023-07-18 03:55:39,094 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 4984 2023-07-18 03:55:39,094 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/f95930cf45f370dfce61fdb9733fd578/4=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/5=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/6=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/7=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/8=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/9=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/10=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/11=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/12=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/13=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/14=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/15=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/16=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/17=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/18=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/19=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/20=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/21=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/22=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/23=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/24=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/25=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/26=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/27=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/28=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/29=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/30=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/31=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/32=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/33=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/34=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/35=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/36=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/37=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/38=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/39=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/40=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/41=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/42=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/43=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/44=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/45=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/46=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/47=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992, lastWalPosition=4984, nbRowKeys=44, nbHFiles=0, heapSize=9152, lastSeqIds={}, endOfFile=false,usedBufferSize=9152] 2023-07-18 03:55:39,099 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:39,100 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,100 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 29 WAL entries eligible for replication 2023-07-18 03:55:39,100 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,100 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/75=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/76=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/77=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/78=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/79=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/80=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/81=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/82=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/83=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/84=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/85=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/86=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/87=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/88=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/89=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/90=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/91=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/92=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/93=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/94=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/95=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/96=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/97=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/98=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/99=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/100=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/101=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/102=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992, lastWalPosition=11492, nbRowKeys=29, nbHFiles=0, heapSize=6032, lastSeqIds={}, endOfFile=false,usedBufferSize=6032] 2023-07-18 03:55:39,100 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39470, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-18 03:55:39,103 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45597] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:39,103 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:39,104 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45597] zookeeper.ReadOnlyZKClient(139): Connect 0x70182e94 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:39,107 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:39,118 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@593afe71, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:39,120 DEBUG [hconnection-0x41a2b9a5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,121 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43310, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,128 DEBUG [hconnection-0x41a2b9a5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,130 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,135 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45597] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:39,211 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 13432 2023-07-18 03:55:39,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceWALReader(162): Read 80 WAL entries eligible for replication 2023-07-18 03:55:39,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 13432 2023-07-18 03:55:39,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/f95930cf45f370dfce61fdb9733fd578/48=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/49=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/50=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/51=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/52=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/53=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/54=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/55=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/56=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/57=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/58=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/59=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/60=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/61=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/62=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/63=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/64=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/65=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/66=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/67=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/68=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/69=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/70=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/71=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/72=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/73=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/74=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/75=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/76=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/77=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/78=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/79=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/80=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/81=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/82=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/83=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/84=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/85=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/86=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/87=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/88=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/89=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/90=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/91=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/92=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/93=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/94=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/95=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/96=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/97=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/98=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/99=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/100=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/101=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/102=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/103=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/104=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/105=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/106=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/107=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/108=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/109=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/110=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/111=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/112=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/113=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/114=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/115=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/116=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/117=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/118=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/119=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/120=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/121=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/122=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/123=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/124=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/125=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/126=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/127=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992, lastWalPosition=13432, nbRowKeys=80, nbHFiles=0, heapSize=16640, lastSeqIds={}, endOfFile=false,usedBufferSize=16640] 2023-07-18 03:55:39,227 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:39,229 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43322, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-18 03:55:39,232 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:39,237 DEBUG [hconnection-0x8094b60-shared-pool-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,240 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33750, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,246 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=39019] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:39,323 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,338 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceWALReader(162): Read 76 WAL entries eligible for replication 2023-07-18 03:55:39,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/f95930cf45f370dfce61fdb9733fd578/128=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/129=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/130=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/131=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/132=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/133=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/134=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/135=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/136=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/137=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/138=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/139=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/140=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/141=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/142=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/143=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/144=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/145=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/146=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/147=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/148=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/149=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/150=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/151=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/152=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/153=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/154=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/155=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/156=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/157=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/158=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/159=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/160=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/161=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/162=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/163=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/164=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/165=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/166=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/167=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/168=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/169=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/170=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/171=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/172=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/173=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/174=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/175=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/176=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/177=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/178=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/179=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/180=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/181=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/182=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/183=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/184=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/185=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/186=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/187=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/188=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/189=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/190=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/191=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/192=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/193=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/194=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/195=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/196=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/197=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/198=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/199=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/200=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/201=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/202=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992, lastWalPosition=21752, nbRowKeys=76, nbHFiles=0, heapSize=15808, lastSeqIds={}, endOfFile=false,usedBufferSize=15808] 2023-07-18 03:55:39,340 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:39,342 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33764, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-18 03:55:39,344 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33473] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:39,345 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33473] zookeeper.ReadOnlyZKClient(139): Connect 0x6e3ea6fe to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:39,351 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@60f94549, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:39,353 DEBUG [hconnection-0x64c6cd8e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,354 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43332, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,360 DEBUG [hconnection-0x64c6cd8e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,361 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33776, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,365 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33473] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:39,432 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,447 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,465 INFO [jenkins-hbase4:33473Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:39,465 INFO [jenkins-hbase4:33473Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 76 2023-07-18 03:55:39,466 INFO [jenkins-hbase4:39019Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:39,466 INFO [jenkins-hbase4:39019Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 180 2023-07-18 03:55:39,468 INFO [jenkins-hbase4:36323Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:39,479 INFO [jenkins-hbase4:45597Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:39,479 INFO [jenkins-hbase4:45597Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 44 2023-07-18 03:55:39,541 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,553 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,651 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,660 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,759 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,771 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,821 DEBUG [hconnection-0x7cf5edca-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,823 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43344, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,842 DEBUG [hconnection-0x7cf5edca-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:39,844 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33778, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:39,850 DEBUG [Listener at localhost/46383] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-18 03:55:39,850 INFO [Listener at localhost/46383] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-18 03:55:39,850 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4be4c0d5 to 127.0.0.1:60939 2023-07-18 03:55:39,850 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,851 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 03:55:39,851 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(257): Found active master hash=795267191, stopped=false 2023-07-18 03:55:39,851 INFO [Listener at localhost/46383] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:39,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:55:39,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:55:39,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:55:39,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:55:39,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:55:39,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:39,856 INFO [Listener at localhost/46383] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 03:55:39,856 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70f6554f to 127.0.0.1:60939 2023-07-18 03:55:39,858 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:39,858 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45597,1689652533808' ***** 2023-07-18 03:55:39,858 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:55:39,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:39,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:39,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:39,858 INFO [RS:0;jenkins-hbase4:45597] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:55:39,859 INFO [RS:0;jenkins-hbase4:45597] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:55:39,859 INFO [RS:0;jenkins-hbase4:45597] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:55:39,859 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(3305): Received CLOSE for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:39,858 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36323,1689652533838' ***** 2023-07-18 03:55:39,859 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:55:39,859 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:55:39,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:39,859 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39019,1689652533864' ***** 2023-07-18 03:55:39,860 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:55:39,860 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33473,1689652533891' ***** 2023-07-18 03:55:39,860 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:55:39,860 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:39,860 INFO [RS:3;jenkins-hbase4:33473] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:55:39,860 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28f10c33 to 127.0.0.1:60939 2023-07-18 03:55:39,860 INFO [RS:3;jenkins-hbase4:33473] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:55:39,860 DEBUG [RS:0;jenkins-hbase4:45597] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,862 INFO [RS:3;jenkins-hbase4:33473] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:55:39,862 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-18 03:55:39,862 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(3305): Received CLOSE for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:39,862 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1478): Online Regions={6596a12435c5c3b7add032ab29aea733=hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.} 2023-07-18 03:55:39,862 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:39,864 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:55:39,864 INFO [RS:1;jenkins-hbase4:36323] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:55:39,864 INFO [RS:2;jenkins-hbase4:39019] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:55:39,865 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:55:39,865 INFO [RS:2;jenkins-hbase4:39019] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:55:39,865 INFO [RS:1;jenkins-hbase4:36323] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:55:39,865 INFO [RS:2;jenkins-hbase4:39019] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:55:39,865 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(3305): Received CLOSE for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:39,865 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:55:39,865 INFO [RS:1;jenkins-hbase4:36323] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:55:39,865 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:39,865 DEBUG [RS:1;jenkins-hbase4:36323] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c7d2585 to 127.0.0.1:60939 2023-07-18 03:55:39,865 DEBUG [RS:1;jenkins-hbase4:36323] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,866 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36323,1689652533838; all regions closed. 2023-07-18 03:55:39,866 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:39,866 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x228710aa to 127.0.0.1:60939 2023-07-18 03:55:39,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6596a12435c5c3b7add032ab29aea733, disabling compactions & flushes 2023-07-18 03:55:39,867 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18a1f13d to 127.0.0.1:60939 2023-07-18 03:55:39,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:39,867 DEBUG [RS:3;jenkins-hbase4:33473] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,867 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-18 03:55:39,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:39,867 DEBUG [RS:2;jenkins-hbase4:39019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. after waiting 0 ms 2023-07-18 03:55:39,867 INFO [RS:2;jenkins-hbase4:39019] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:55:39,867 INFO [RS:2;jenkins-hbase4:39019] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:55:39,867 INFO [RS:2;jenkins-hbase4:39019] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:55:39,867 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 03:55:39,868 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1504): Waiting on 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:39,867 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1478): Online Regions={d59bee2bde49e7e5abb27771a5bfd3f4=t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.} 2023-07-18 03:55:39,869 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1504): Waiting on d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:39,869 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:39,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6596a12435c5c3b7add032ab29aea733 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-18 03:55:39,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d59bee2bde49e7e5abb27771a5bfd3f4, disabling compactions & flushes 2023-07-18 03:55:39,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6d104435467e1000121c02223fd064d0, disabling compactions & flushes 2023-07-18 03:55:39,876 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-18 03:55:39,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:39,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:39,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:39,878 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:55:39,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. after waiting 0 ms 2023-07-18 03:55:39,876 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 6d104435467e1000121c02223fd064d0=t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.} 2023-07-18 03:55:39,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:39,879 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1504): Waiting on 1588230740, 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:39,879 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:55:39,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6d104435467e1000121c02223fd064d0 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-18 03:55:39,880 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:39,879 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:55:39,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. after waiting 0 ms 2023-07-18 03:55:39,880 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:55:39,883 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:55:39,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:39,885 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-18 03:55:39,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d59bee2bde49e7e5abb27771a5bfd3f4 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-18 03:55:39,888 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:39,895 DEBUG [RS:1;jenkins-hbase4:36323] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:55:39,895 INFO [RS:1;jenkins-hbase4:36323] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36323%2C1689652533838:(num 1689652534580) 2023-07-18 03:55:39,895 DEBUG [RS:1;jenkins-hbase4:36323] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:39,895 INFO [RS:1;jenkins-hbase4:36323] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:39,895 INFO [RS:1;jenkins-hbase4:36323] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 03:55:39,895 INFO [RS:1;jenkins-hbase4:36323] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:55:39,896 INFO [RS:1;jenkins-hbase4:36323] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:55:39,896 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:39,896 INFO [RS:1;jenkins-hbase4:36323] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:55:39,897 INFO [RS:1;jenkins-hbase4:36323] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36323 2023-07-18 03:55:39,909 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:39,909 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,909 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:39,910 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0017, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,910 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,910 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:39,909 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:39,909 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36323,1689652533838 2023-07-18 03:55:39,909 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,909 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0018, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,910 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,910 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:39,911 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:39,913 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36323,1689652533838] 2023-07-18 03:55:39,913 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36323,1689652533838; numProcessing=1 2023-07-18 03:55:39,914 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,36323,1689652533838 already deleted, retry=false 2023-07-18 03:55:39,914 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36323,1689652533838 expired; onlineServers=3 2023-07-18 03:55:39,950 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:39,950 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:39,950 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:39,955 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:39,999 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:39,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/.tmp/cf1/3d270ab02d934440a4e65c4aa122741c 2023-07-18 03:55:40,003 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/.tmp/cf1/3d270ab02d934440a4e65c4aa122741c as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/3d270ab02d934440a4e65c4aa122741c 2023-07-18 03:55:40,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/3d270ab02d934440a4e65c4aa122741c, entries=200, sequenceid=7, filesize=12.2 K 2023-07-18 03:55:40,057 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,057 INFO [RS:1;jenkins-hbase4:36323] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36323,1689652533838; zookeeper connection closed. 2023-07-18 03:55:40,057 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:36323-0x1017691f55f0009, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,058 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@47f5f47a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@47f5f47a 2023-07-18 03:55:40,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for d59bee2bde49e7e5abb27771a5bfd3f4 in 177ms, sequenceid=7, compaction requested=false 2023-07-18 03:55:40,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-18 03:55:40,071 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1504): Waiting on 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:40,071 DEBUG [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1504): Waiting on d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:40,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:40,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:55:40,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:40,079 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1504): Waiting on 1588230740, 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:40,106 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,109 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,213 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,218 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,271 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33473,1689652533891; all regions closed. 2023-07-18 03:55:40,271 DEBUG [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1504): Waiting on 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:40,278 DEBUG [RS:3;jenkins-hbase4:33473] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:55:40,278 INFO [RS:3;jenkins-hbase4:33473] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33473%2C1689652533891:(num 1689652534579) 2023-07-18 03:55:40,278 DEBUG [RS:3;jenkins-hbase4:33473] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,278 INFO [RS:3;jenkins-hbase4:33473] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:40,278 INFO [RS:3;jenkins-hbase4:33473] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 03:55:40,279 INFO [RS:3;jenkins-hbase4:33473] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:55:40,279 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:40,279 INFO [RS:3;jenkins-hbase4:33473] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:55:40,279 INFO [RS:3;jenkins-hbase4:33473] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:55:40,279 DEBUG [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1504): Waiting on 1588230740, 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:40,280 DEBUG [RS:3;jenkins-hbase4:33473] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6e3ea6fe to 127.0.0.1:60939 2023-07-18 03:55:40,280 DEBUG [RS:3;jenkins-hbase4:33473] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,281 INFO [RS:3;jenkins-hbase4:33473] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33473 2023-07-18 03:55:40,289 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:40,289 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:40,289 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,33473,1689652533891 2023-07-18 03:55:40,289 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,289 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0018, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,289 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0017, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,289 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:40,289 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:40,294 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33473,1689652533891] 2023-07-18 03:55:40,294 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33473,1689652533891; numProcessing=2 2023-07-18 03:55:40,295 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33473,1689652533891 already deleted, retry=false 2023-07-18 03:55:40,295 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33473,1689652533891 expired; onlineServers=2 2023-07-18 03:55:40,322 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,327 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/.tmp/cf1/f7b510ff27dc4e3286f94f9b58c2ef73 2023-07-18 03:55:40,394 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/info/aa1e6b16427d4dbfacbeab6bd7ed6ed4 2023-07-18 03:55:40,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/.tmp/info/7d27b66e1d264872bc50ba77246226b1 2023-07-18 03:55:40,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/.tmp/cf1/f7b510ff27dc4e3286f94f9b58c2ef73 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/f7b510ff27dc4e3286f94f9b58c2ef73 2023-07-18 03:55:40,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/.tmp/info/7d27b66e1d264872bc50ba77246226b1 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info/7d27b66e1d264872bc50ba77246226b1 2023-07-18 03:55:40,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/f7b510ff27dc4e3286f94f9b58c2ef73, entries=100, sequenceid=7, filesize=8.3 K 2023-07-18 03:55:40,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 6d104435467e1000121c02223fd064d0 in 536ms, sequenceid=7, compaction requested=false 2023-07-18 03:55:40,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info/7d27b66e1d264872bc50ba77246226b1, entries=2, sequenceid=6, filesize=4.8 K 2023-07-18 03:55:40,420 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6596a12435c5c3b7add032ab29aea733 in 550ms, sequenceid=6, compaction requested=false 2023-07-18 03:55:40,433 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/table/e35dec1443364b0288f9553a65a94860 2023-07-18 03:55:40,436 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-18 03:55:40,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-18 03:55:40,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:40,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:55:40,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:40,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:40,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:55:40,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:40,440 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,443 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/info/aa1e6b16427d4dbfacbeab6bd7ed6ed4 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/aa1e6b16427d4dbfacbeab6bd7ed6ed4 2023-07-18 03:55:40,448 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/aa1e6b16427d4dbfacbeab6bd7ed6ed4, entries=30, sequenceid=19, filesize=8.1 K 2023-07-18 03:55:40,449 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/table/e35dec1443364b0288f9553a65a94860 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table/e35dec1443364b0288f9553a65a94860 2023-07-18 03:55:40,455 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table/e35dec1443364b0288f9553a65a94860, entries=6, sequenceid=19, filesize=4.8 K 2023-07-18 03:55:40,456 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 573ms, sequenceid=19, compaction requested=false 2023-07-18 03:55:40,465 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-18 03:55:40,466 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 03:55:40,466 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:55:40,466 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:55:40,466 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 03:55:40,471 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45597,1689652533808; all regions closed. 2023-07-18 03:55:40,477 DEBUG [RS:0;jenkins-hbase4:45597] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:55:40,477 INFO [RS:0;jenkins-hbase4:45597] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45597%2C1689652533808:(num 1689652534588) 2023-07-18 03:55:40,477 DEBUG [RS:0;jenkins-hbase4:45597] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,477 INFO [RS:0;jenkins-hbase4:45597] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:40,478 INFO [RS:0;jenkins-hbase4:45597] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 03:55:40,478 INFO [RS:0;jenkins-hbase4:45597] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:55:40,478 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:40,478 INFO [RS:0;jenkins-hbase4:45597] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:55:40,478 INFO [RS:0;jenkins-hbase4:45597] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:55:40,478 DEBUG [RS:0;jenkins-hbase4:45597] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70182e94 to 127.0.0.1:60939 2023-07-18 03:55:40,478 DEBUG [RS:0;jenkins-hbase4:45597] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,479 INFO [RS:0;jenkins-hbase4:45597] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45597 2023-07-18 03:55:40,480 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39019,1689652533864; all regions closed. 2023-07-18 03:55:40,481 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,481 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:40,481 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0017, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,481 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45597,1689652533808 2023-07-18 03:55:40,481 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0018, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,481 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:40,481 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45597,1689652533808] 2023-07-18 03:55:40,481 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:40,482 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45597,1689652533808; numProcessing=3 2023-07-18 03:55:40,485 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45597,1689652533808 already deleted, retry=false 2023-07-18 03:55:40,485 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45597,1689652533808 expired; onlineServers=1 2023-07-18 03:55:40,487 DEBUG [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:55:40,487 INFO [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39019%2C1689652533864.meta:.meta(num 1689652534775) 2023-07-18 03:55:40,493 DEBUG [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:55:40,493 INFO [RS:2;jenkins-hbase4:39019] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39019%2C1689652533864:(num 1689652534578) 2023-07-18 03:55:40,493 DEBUG [RS:2;jenkins-hbase4:39019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,493 INFO [RS:2;jenkins-hbase4:39019] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:40,493 INFO [RS:2;jenkins-hbase4:39019] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-18 03:55:40,494 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:40,494 DEBUG [RS:2;jenkins-hbase4:39019] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x59049db1 to 127.0.0.1:60939 2023-07-18 03:55:40,494 DEBUG [RS:2;jenkins-hbase4:39019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,495 INFO [RS:2;jenkins-hbase4:39019] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39019 2023-07-18 03:55:40,543 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,547 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,584 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,584 INFO [RS:0;jenkins-hbase4:45597] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45597,1689652533808; zookeeper connection closed. 2023-07-18 03:55:40,584 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45597-0x1017691f55f0008, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,585 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1e6df024] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1e6df024 2023-07-18 03:55:40,586 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,39019,1689652533864 2023-07-18 03:55:40,586 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0018, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,586 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,586 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0017, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:40,586 ERROR [Listener at localhost/46383-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@109845f3 rejected from java.util.concurrent.ThreadPoolExecutor@14a7c338[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:603) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-07-18 03:55:40,586 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:40,586 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:55:40,588 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39019,1689652533864] 2023-07-18 03:55:40,588 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39019,1689652533864; numProcessing=4 2023-07-18 03:55:40,589 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,39019,1689652533864 already deleted, retry=false 2023-07-18 03:55:40,589 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39019,1689652533864 expired; onlineServers=0 2023-07-18 03:55:40,589 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33247,1689652533764' ***** 2023-07-18 03:55:40,589 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 03:55:40,590 DEBUG [M:0;jenkins-hbase4:33247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b594143, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:40,590 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33247,1689652533764 2023-07-18 03:55:40,590 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33247,1689652533764; all regions closed. 2023-07-18 03:55:40,590 DEBUG [M:0;jenkins-hbase4:33247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:40,591 DEBUG [M:0;jenkins-hbase4:33247] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 03:55:40,591 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 03:55:40,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652534152] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652534152,5,FailOnTimeoutGroup] 2023-07-18 03:55:40,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652534152] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652534152,5,FailOnTimeoutGroup] 2023-07-18 03:55:40,591 DEBUG [M:0;jenkins-hbase4:33247] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 03:55:40,593 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-18 03:55:40,593 INFO [M:0;jenkins-hbase4:33247] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 03:55:40,593 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:40,593 INFO [M:0;jenkins-hbase4:33247] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 03:55:40,593 INFO [M:0;jenkins-hbase4:33247] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 03:55:40,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:40,593 DEBUG [M:0;jenkins-hbase4:33247] master.HMaster(1512): Stopping service threads 2023-07-18 03:55:40,593 INFO [M:0;jenkins-hbase4:33247] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 03:55:40,594 ERROR [M:0;jenkins-hbase4:33247] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-18 03:55:40,595 INFO [M:0;jenkins-hbase4:33247] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 03:55:40,595 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 03:55:40,595 DEBUG [M:0;jenkins-hbase4:33247] zookeeper.ZKUtil(398): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-18 03:55:40,595 WARN [M:0;jenkins-hbase4:33247] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 03:55:40,595 INFO [M:0;jenkins-hbase4:33247] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 03:55:40,596 INFO [M:0;jenkins-hbase4:33247] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 03:55:40,597 DEBUG [M:0;jenkins-hbase4:33247] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:55:40,597 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:40,597 DEBUG [M:0;jenkins-hbase4:33247] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:40,597 DEBUG [M:0;jenkins-hbase4:33247] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:55:40,597 DEBUG [M:0;jenkins-hbase4:33247] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:40,597 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.39 KB heapSize=65.18 KB 2023-07-18 03:55:40,613 INFO [M:0;jenkins-hbase4:33247] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.39 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e8074e048f744f8db8f92f79e670bb4b 2023-07-18 03:55:40,619 DEBUG [M:0;jenkins-hbase4:33247] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e8074e048f744f8db8f92f79e670bb4b as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e8074e048f744f8db8f92f79e670bb4b 2023-07-18 03:55:40,625 INFO [M:0;jenkins-hbase4:33247] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e8074e048f744f8db8f92f79e670bb4b, entries=14, sequenceid=116, filesize=8.1 K 2023-07-18 03:55:40,626 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegion(2948): Finished flush of dataSize ~55.39 KB/56720, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=116, compaction requested=false 2023-07-18 03:55:40,628 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:40,628 DEBUG [M:0;jenkins-hbase4:33247] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:40,636 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:40,636 INFO [M:0;jenkins-hbase4:33247] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 03:55:40,637 INFO [M:0;jenkins-hbase4:33247] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33247 2023-07-18 03:55:40,639 DEBUG [M:0;jenkins-hbase4:33247] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,33247,1689652533764 already deleted, retry=false 2023-07-18 03:55:40,648 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,651 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,754 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,758 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,759 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,759 INFO [RS:3;jenkins-hbase4:33473] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33473,1689652533891; zookeeper connection closed. 2023-07-18 03:55:40,759 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:33473-0x1017691f55f000b, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,760 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@71d836] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@71d836 2023-07-18 03:55:40,861 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,868 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:40,869 INFO [jenkins-hbase4:36729Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 100, current progress: walGroup [jenkins-hbase4.apache.org%2C36729%2C1689652529501]: currently replicating from: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992 at position: 11492 2023-07-18 03:55:40,869 INFO [jenkins-hbase4:42337Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 200, current progress: walGroup [jenkins-hbase4.apache.org%2C42337%2C1689652529544]: currently replicating from: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 at position: 21752 2023-07-18 03:55:40,959 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,959 INFO [M:0;jenkins-hbase4:33247] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33247,1689652533764; zookeeper connection closed. 2023-07-18 03:55:40,959 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:33247-0x1017691f55f0007, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:40,966 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 11492 2023-07-18 03:55:40,973 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:41,059 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,059 INFO [RS:2;jenkins-hbase4:39019] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39019,1689652533864; zookeeper connection closed. 2023-07-18 03:55:41,060 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:39019-0x1017691f55f000a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,063 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3a6917a5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3a6917a5 2023-07-18 03:55:41,064 INFO [Listener at localhost/46383] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-18 03:55:41,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 14838 2023-07-18 03:55:41,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 03:55:41,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 14838 2023-07-18 03:55:41,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992, lastWalPosition=14838, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-18 03:55:41,074 INFO [SinkThread-3] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 03:55:41,076 WARN [SinkThread-3] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:55:41,077 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:41,079 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-18 03:55:41,091 INFO [Listener at localhost/46383] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-18 03:55:41,092 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x448b0fe6 to 127.0.0.1:60939 2023-07-18 03:55:41,092 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,092 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 03:55:41,092 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(257): Found active master hash=610686535, stopped=false 2023-07-18 03:55:41,092 INFO [Listener at localhost/46383] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:41,094 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 03:55:41,094 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 03:55:41,094 INFO [Listener at localhost/46383] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 03:55:41,094 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3886878b to 127.0.0.1:60939 2023-07-18 03:55:41,094 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,094 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 03:55:41,096 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:41,098 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36729,1689652529501' ***** 2023-07-18 03:55:41,099 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:55:41,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:55:41,099 INFO [RS:0;jenkins-hbase4:36729] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:55:41,099 INFO [RS:0;jenkins-hbase4:36729] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:55:41,099 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42337,1689652529544' ***** 2023-07-18 03:55:41,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:55:41,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:55:41,099 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:55:41,099 INFO [RS:0;jenkins-hbase4:36729] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:55:41,100 INFO [RS:1;jenkins-hbase4:42337] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:55:41,099 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:55:41,100 INFO [RS:1;jenkins-hbase4:42337] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:55:41,100 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(3305): Received CLOSE for f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:41,101 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:55:41,101 INFO [RS:1;jenkins-hbase4:42337] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:55:41,102 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(3305): Received CLOSE for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:41,102 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(3305): Received CLOSE for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:55:41,102 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:41,102 DEBUG [RS:0;jenkins-hbase4:36729] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b39d69d to 127.0.0.1:60939 2023-07-18 03:55:41,102 DEBUG [RS:0;jenkins-hbase4:36729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,102 INFO [RS:0;jenkins-hbase4:36729] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:55:41,102 INFO [RS:0;jenkins-hbase4:36729] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:55:41,102 INFO [RS:0;jenkins-hbase4:36729] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:55:41,102 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 03:55:41,102 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:41,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1f3df17ddf51de240de231140850f62, disabling compactions & flushes 2023-07-18 03:55:41,105 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-18 03:55:41,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:41,105 DEBUG [RS:1;jenkins-hbase4:42337] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x23232f5a to 127.0.0.1:60939 2023-07-18 03:55:41,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:41,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:55:41,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. after waiting 0 ms 2023-07-18 03:55:41,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f95930cf45f370dfce61fdb9733fd578, disabling compactions & flushes 2023-07-18 03:55:41,105 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1478): Online Regions={f1f3df17ddf51de240de231140850f62=t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62., 1588230740=hbase:meta,,1.1588230740, 12eb339fe9277c226d5f3d9d08310c02=hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.} 2023-07-18 03:55:41,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:41,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:41,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:55:41,106 DEBUG [RS:1;jenkins-hbase4:42337] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,107 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:55:41,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f1f3df17ddf51de240de231140850f62 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-18 03:55:41,106 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1504): Waiting on 12eb339fe9277c226d5f3d9d08310c02, 1588230740, f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:41,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:41,107 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:55:41,107 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-18 03:55:41,107 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:55:41,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. after waiting 0 ms 2023-07-18 03:55:41,107 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1478): Online Regions={f95930cf45f370dfce61fdb9733fd578=t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.} 2023-07-18 03:55:41,107 DEBUG [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1504): Waiting on f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:55:41,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:41,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f95930cf45f370dfce61fdb9733fd578 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-18 03:55:41,107 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-18 03:55:41,147 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/info/51c9bb8f378b4fb4abb67c058897fc76 2023-07-18 03:55:41,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/cf1/36e6fc5b788b41c19d8786bebda95cb2 2023-07-18 03:55:41,149 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:41,153 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:41,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/cf1/100e4311365a4b829c861a164a543574 2023-07-18 03:55:41,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36e6fc5b788b41c19d8786bebda95cb2 2023-07-18 03:55:41,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 100e4311365a4b829c861a164a543574 2023-07-18 03:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 15079 2023-07-18 03:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 03:55:41,184 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 15079 2023-07-18 03:55:41,186 INFO [SinkThread-4] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 03:55:41,186 WARN [SinkThread-4] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:55:41,186 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:41,202 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-07-18 03:55:41,202 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 03:55:41,202 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-07-18 03:55:41,202 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/f95930cf45f370dfce61fdb9733fd578/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992, lastWalPosition=28643, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-18 03:55:41,213 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/rep_barrier/d8c0ee8d8ad841798f467b6bfed755f0 2023-07-18 03:55:41,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/norep/4f2be17687e34c4f9789b573b25ee02d 2023-07-18 03:55:41,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f2be17687e34c4f9789b573b25ee02d 2023-07-18 03:55:41,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/cf1/100e4311365a4b829c861a164a543574 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1/100e4311365a4b829c861a164a543574 2023-07-18 03:55:41,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 100e4311365a4b829c861a164a543574 2023-07-18 03:55:41,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1/100e4311365a4b829c861a164a543574, entries=200, sequenceid=206, filesize=11.9 K 2023-07-18 03:55:41,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/norep/4f2be17687e34c4f9789b573b25ee02d as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep/4f2be17687e34c4f9789b573b25ee02d 2023-07-18 03:55:41,264 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/table/f40694dec6b74e12ae0468b21b6767ee 2023-07-18 03:55:41,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f2be17687e34c4f9789b573b25ee02d 2023-07-18 03:55:41,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep/4f2be17687e34c4f9789b573b25ee02d, entries=101, sequenceid=206, filesize=8.3 K 2023-07-18 03:55:41,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for f95930cf45f370dfce61fdb9733fd578 in 163ms, sequenceid=206, compaction requested=false 2023-07-18 03:55:41,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/info/51c9bb8f378b4fb4abb67c058897fc76 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info/51c9bb8f378b4fb4abb67c058897fc76 2023-07-18 03:55:41,280 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info/51c9bb8f378b4fb4abb67c058897fc76, entries=30, sequenceid=19, filesize=8.1 K 2023-07-18 03:55:41,281 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/rep_barrier/d8c0ee8d8ad841798f467b6bfed755f0 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier/d8c0ee8d8ad841798f467b6bfed755f0 2023-07-18 03:55:41,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-18 03:55:41,290 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 15079 2023-07-18 03:55:41,291 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:41,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f95930cf45f370dfce61fdb9733fd578: 2023-07-18 03:55:41,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:55:41,293 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier/d8c0ee8d8ad841798f467b6bfed755f0, entries=2, sequenceid=19, filesize=4.9 K 2023-07-18 03:55:41,294 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/table/f40694dec6b74e12ae0468b21b6767ee as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table/f40694dec6b74e12ae0468b21b6767ee 2023-07-18 03:55:41,300 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table/f40694dec6b74e12ae0468b21b6767ee, entries=6, sequenceid=19, filesize=4.8 K 2023-07-18 03:55:41,301 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 194ms, sequenceid=19, compaction requested=false 2023-07-18 03:55:41,307 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42337,1689652529544; all regions closed. 2023-07-18 03:55:41,308 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1504): Waiting on 12eb339fe9277c226d5f3d9d08310c02, 1588230740, f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:41,315 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-18 03:55:41,316 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 03:55:41,317 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:55:41,317 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:55:41,317 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 03:55:41,317 DEBUG [RS:1;jenkins-hbase4:42337] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs 2023-07-18 03:55:41,317 INFO [RS:1;jenkins-hbase4:42337] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42337%2C1689652529544:(num 1689652530992) 2023-07-18 03:55:41,317 DEBUG [RS:1;jenkins-hbase4:42337] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,317 INFO [RS:1;jenkins-hbase4:42337] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:41,318 INFO [RS:1;jenkins-hbase4:42337] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-18 03:55:41,318 INFO [RS:1;jenkins-hbase4:42337] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:55:41,318 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:41,318 INFO [RS:1;jenkins-hbase4:42337] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:55:41,318 INFO [RS:1;jenkins-hbase4:42337] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:55:41,318 INFO [RS:1;jenkins-hbase4:42337] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 03:55:41,320 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.ProtobufLogReader(425): Encountered a malformed edit, seeking back to last good position in file, from 29266 to 29258 java.io.EOFException: Invalid PB, EOF? Ignoring; originalPosition=29258, currentPosition=29266 at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:354) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:95) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:83) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.readNextEntryAndRecordReaderPosition(WALEntryStream.java:259) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:173) at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:102) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.readWALEntries(ReplicationSourceWALReader.java:230) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:157) Caused by: org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: encoded_region_name, table_name, log_sequence_number, write_time at org.apache.hbase.thirdparty.com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:79) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.checkMessageInitialized(AbstractParser.java:68) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:232) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:237) at org.apache.hbase.thirdparty.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.parseDelimitedFrom(ProtobufUtil.java:3576) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:348) ... 7 more 2023-07-18 03:55:41,321 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 was moved to hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 2023-07-18 03:55:41,324 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 2023-07-18 03:55:41,324 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 03:55:41,388 INFO [SinkThread-5] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 03:55:41,389 WARN [SinkThread-5] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:55:41,389 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:41,396 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 15079 2023-07-18 03:55:41,501 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 15079 2023-07-18 03:55:41,508 DEBUG [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1504): Waiting on 12eb339fe9277c226d5f3d9d08310c02, f1f3df17ddf51de240de231140850f62 2023-07-18 03:55:41,606 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 15079 2023-07-18 03:55:41,625 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/norep/9f0cf58662854a048aad2485229f1a4f 2023-07-18 03:55:41,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9f0cf58662854a048aad2485229f1a4f 2023-07-18 03:55:41,634 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/cf1/36e6fc5b788b41c19d8786bebda95cb2 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1/36e6fc5b788b41c19d8786bebda95cb2 2023-07-18 03:55:41,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36e6fc5b788b41c19d8786bebda95cb2 2023-07-18 03:55:41,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1/36e6fc5b788b41c19d8786bebda95cb2, entries=100, sequenceid=106, filesize=8.2 K 2023-07-18 03:55:41,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/norep/9f0cf58662854a048aad2485229f1a4f as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep/9f0cf58662854a048aad2485229f1a4f 2023-07-18 03:55:41,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9f0cf58662854a048aad2485229f1a4f 2023-07-18 03:55:41,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep/9f0cf58662854a048aad2485229f1a4f, entries=51, sequenceid=106, filesize=6.5 K 2023-07-18 03:55:41,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for f1f3df17ddf51de240de231140850f62 in 548ms, sequenceid=106, compaction requested=false 2023-07-18 03:55:41,661 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0017, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,661 DEBUG [RS:1;jenkins-hbase4:42337] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1bafb0f2 to 127.0.0.1:60939 2023-07-18 03:55:41,661 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0017, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,661 DEBUG [RS:1;jenkins-hbase4:42337] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,664 INFO [RS:1;jenkins-hbase4:42337] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1 terminated 2023-07-18 03:55:41,664 INFO [RS:1;jenkins-hbase4:42337] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42337 2023-07-18 03:55:41,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-18 03:55:41,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:41,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1f3df17ddf51de240de231140850f62: 2023-07-18 03:55:41,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:55:41,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 12eb339fe9277c226d5f3d9d08310c02, disabling compactions & flushes 2023-07-18 03:55:41,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:41,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:41,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. after waiting 0 ms 2023-07-18 03:55:41,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:41,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 12eb339fe9277c226d5f3d9d08310c02 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-18 03:55:41,670 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:41,670 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:41,670 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:55:41,670 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:55:41,670 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:55:41,671 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42337,1689652529544] 2023-07-18 03:55:41,671 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42337,1689652529544; numProcessing=1 2023-07-18 03:55:41,672 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42337,1689652529544 already deleted, retry=false 2023-07-18 03:55:41,672 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42337,1689652529544 expired; onlineServers=1 2023-07-18 03:55:41,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/.tmp/info/3759eb675381438895a87e71f03c0574 2023-07-18 03:55:41,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/.tmp/info/3759eb675381438895a87e71f03c0574 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info/3759eb675381438895a87e71f03c0574 2023-07-18 03:55:41,690 INFO [SinkThread-6] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 03:55:41,691 WARN [SinkThread-6] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:55:41,691 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 03:55:41,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info/3759eb675381438895a87e71f03c0574, entries=2, sequenceid=6, filesize=4.8 K 2023-07-18 03:55:41,696 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 12eb339fe9277c226d5f3d9d08310c02 in 26ms, sequenceid=6, compaction requested=false 2023-07-18 03:55:41,701 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-18 03:55:41,702 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:41,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 12eb339fe9277c226d5f3d9d08310c02: 2023-07-18 03:55:41,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:55:41,708 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36729,1689652529501; all regions closed. 2023-07-18 03:55:41,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] wal.ProtobufLogReader(420): EOF at position 16520 2023-07-18 03:55:41,714 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 03:55:41,715 DEBUG [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs 2023-07-18 03:55:41,715 INFO [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36729%2C1689652529501.meta:.meta(num 1689652531137) 2023-07-18 03:55:41,721 DEBUG [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs 2023-07-18 03:55:41,721 INFO [RS:0;jenkins-hbase4:36729] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36729%2C1689652529501:(num 1689652530992) 2023-07-18 03:55:41,721 DEBUG [RS:0;jenkins-hbase4:36729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,721 INFO [RS:0;jenkins-hbase4:36729] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:55:41,721 INFO [RS:0;jenkins-hbase4:36729] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 03:55:41,721 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:41,721 INFO [RS:0;jenkins-hbase4:36729] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 03:55:41,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0018, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,865 DEBUG [RS:0;jenkins-hbase4:36729] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x620c749e to 127.0.0.1:60939 2023-07-18 03:55:41,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0018, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,865 DEBUG [RS:0;jenkins-hbase4:36729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,965 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,965 INFO [RS:1;jenkins-hbase4:42337] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42337,1689652529544; zookeeper connection closed. 2023-07-18 03:55:41,965 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:42337-0x1017691f55f0002, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:41,966 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@79ae0718] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@79ae0718 2023-07-18 03:55:41,968 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1] regionserver.HBaseInterClusterReplicationEndpoint(243): Since we are unable to replicate [Source for peer 1]: Interrupted while sleeping between retries 2023-07-18 03:55:41,968 INFO [RS:0;jenkins-hbase4:36729] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1 terminated 2023-07-18 03:55:41,969 INFO [RS:0;jenkins-hbase4:36729] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36729 2023-07-18 03:55:41,971 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:41,971 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:55:41,972 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36729,1689652529501] 2023-07-18 03:55:41,972 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36729,1689652529501; numProcessing=2 2023-07-18 03:55:41,973 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,36729,1689652529501 already deleted, retry=false 2023-07-18 03:55:41,973 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36729,1689652529501 expired; onlineServers=0 2023-07-18 03:55:41,973 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41639,1689652528089' ***** 2023-07-18 03:55:41,973 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 03:55:41,974 DEBUG [M:0;jenkins-hbase4:41639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@655b5c7c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:41,974 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41639,1689652528089 2023-07-18 03:55:41,974 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41639,1689652528089; all regions closed. 2023-07-18 03:55:41,974 DEBUG [M:0;jenkins-hbase4:41639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:41,974 DEBUG [M:0;jenkins-hbase4:41639] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 03:55:41,975 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 03:55:41,975 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652530649] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652530649,5,FailOnTimeoutGroup] 2023-07-18 03:55:41,975 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652530650] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652530650,5,FailOnTimeoutGroup] 2023-07-18 03:55:41,975 DEBUG [M:0;jenkins-hbase4:41639] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 03:55:41,977 INFO [M:0;jenkins-hbase4:41639] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 03:55:41,977 INFO [M:0;jenkins-hbase4:41639] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 03:55:41,977 INFO [M:0;jenkins-hbase4:41639] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 03:55:41,977 DEBUG [M:0;jenkins-hbase4:41639] master.HMaster(1512): Stopping service threads 2023-07-18 03:55:41,977 INFO [M:0;jenkins-hbase4:41639] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 03:55:41,977 ERROR [M:0;jenkins-hbase4:41639] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-18 03:55:41,978 INFO [M:0;jenkins-hbase4:41639] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 03:55:41,978 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 03:55:41,979 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-18 03:55:41,979 DEBUG [M:0;jenkins-hbase4:41639] zookeeper.ZKUtil(398): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-18 03:55:41,979 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:55:41,979 WARN [M:0;jenkins-hbase4:41639] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 03:55:41,979 INFO [M:0;jenkins-hbase4:41639] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 03:55:41,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:55:41,980 INFO [M:0;jenkins-hbase4:41639] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 03:55:41,980 DEBUG [M:0;jenkins-hbase4:41639] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:55:41,981 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:41,981 DEBUG [M:0;jenkins-hbase4:41639] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:41,981 DEBUG [M:0;jenkins-hbase4:41639] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:55:41,981 DEBUG [M:0;jenkins-hbase4:41639] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:41,981 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.03 KB heapSize=72.60 KB 2023-07-18 03:55:41,998 INFO [M:0;jenkins-hbase4:41639] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.03 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a594b5d4016e4d97b0d35b24053aa8e6 2023-07-18 03:55:42,004 DEBUG [M:0;jenkins-hbase4:41639] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a594b5d4016e4d97b0d35b24053aa8e6 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a594b5d4016e4d97b0d35b24053aa8e6 2023-07-18 03:55:42,009 INFO [M:0;jenkins-hbase4:41639] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a594b5d4016e4d97b0d35b24053aa8e6, entries=17, sequenceid=136, filesize=8.5 K 2023-07-18 03:55:42,010 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegion(2948): Finished flush of dataSize ~61.03 KB/62496, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=136, compaction requested=false 2023-07-18 03:55:42,015 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:55:42,015 DEBUG [M:0;jenkins-hbase4:41639] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:42,018 INFO [M:0;jenkins-hbase4:41639] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 03:55:42,018 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:55:42,019 INFO [M:0;jenkins-hbase4:41639] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41639 2023-07-18 03:55:42,020 DEBUG [M:0;jenkins-hbase4:41639] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,41639,1689652528089 already deleted, retry=false 2023-07-18 03:55:42,166 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:42,166 INFO [M:0;jenkins-hbase4:41639] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41639,1689652528089; zookeeper connection closed. 2023-07-18 03:55:42,166 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): master:41639-0x1017691f55f0000, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:42,240 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:55:42,266 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:42,266 INFO [RS:0;jenkins-hbase4:36729] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36729,1689652529501; zookeeper connection closed. 2023-07-18 03:55:42,266 DEBUG [Listener at localhost/38987-EventThread] zookeeper.ZKWatcher(600): regionserver:36729-0x1017691f55f0001, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:55:42,266 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@55628c68] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@55628c68 2023-07-18 03:55:42,267 INFO [Listener at localhost/46383] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-18 03:55:42,269 INFO [Listener at localhost/46383] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:42,269 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:42,269 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:42,269 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:42,270 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:42,270 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:42,270 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:42,271 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40805 2023-07-18 03:55:42,271 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:42,273 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:42,274 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40805 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:42,278 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:408050x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:42,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40805-0x1017691f55f001c connected 2023-07-18 03:55:42,282 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:42,282 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:42,321 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:42,322 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40805 2023-07-18 03:55:42,323 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40805 2023-07-18 03:55:42,323 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40805 2023-07-18 03:55:42,324 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40805 2023-07-18 03:55:42,324 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40805 2023-07-18 03:55:42,324 INFO [Listener at localhost/46383] master.HMaster(444): hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb, hbase.cluster.distributed=false 2023-07-18 03:55:42,326 DEBUG [pool-440-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-18 03:55:42,338 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:55:42,339 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:42,339 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:42,339 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:55:42,339 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:55:42,339 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:55:42,339 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:55:42,340 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40667 2023-07-18 03:55:42,341 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:55:42,342 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:55:42,343 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:42,345 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:42,346 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40667 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:42,350 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:406670x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:42,351 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40667-0x1017691f55f001d connected 2023-07-18 03:55:42,351 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:55:42,352 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:55:42,352 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:55:42,353 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40667 2023-07-18 03:55:42,353 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40667 2023-07-18 03:55:42,355 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40667 2023-07-18 03:55:42,358 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40667 2023-07-18 03:55:42,358 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40667 2023-07-18 03:55:42,360 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,361 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 03:55:42,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,363 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:42,363 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:55:42,363 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:42,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 03:55:42,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 03:55:42,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,40805,1689652542268 from backup master directory 2023-07-18 03:55:42,367 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,367 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 03:55:42,367 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:42,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:55:42,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7ac44eb7 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:42,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18a08ed0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:42,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:55:42,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 03:55:42,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:42,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764 to hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764-dead as it is dead 2023-07-18 03:55:42,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764-dead/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 2023-07-18 03:55:42,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764-dead/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 after 4ms 2023-07-18 03:55:42,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764-dead/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 to hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 2023-07-18 03:55:42,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,33247,1689652533764-dead 2023-07-18 03:55:42,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40805%2C1689652542268, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/oldWALs, maxLogs=10 2023-07-18 03:55:42,446 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:42,447 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:55:42,448 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:42,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 2023-07-18 03:55:42,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK]] 2023-07-18 03:55:42,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:42,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:42,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:42,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:42,457 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:42,458 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 03:55:42,459 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 03:55:42,465 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e8074e048f744f8db8f92f79e670bb4b 2023-07-18 03:55:42,466 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:42,467 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-18 03:55:42,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 2023-07-18 03:55:42,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 2023-07-18 03:55:42,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33247%2C1689652533764.1689652534035 2023-07-18 03:55:42,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:55:42,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-18 03:55:42,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59265270, jitterRate=-0.11687865853309631}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:55:42,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:55:42,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 03:55:42,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 03:55:42,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 03:55:42,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 03:55:42,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 03:55:42,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-18 03:55:42,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-18 03:55:42,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-18 03:55:42,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-18 03:55:42,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-18 03:55:42,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-18 03:55:42,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 19 msec 2023-07-18 03:55:42,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 03:55:42,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-18 03:55:42,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39019,1689652533864, table=hbase:meta, region=1588230740 2023-07-18 03:55:42,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-18 03:55:42,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,36323,1689652533838 already deleted, retry=false 2023-07-18 03:55:42,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,36323,1689652533838 on jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,36323,1689652533838, splitWal=true, meta=false 2023-07-18 03:55:42,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,36323,1689652533838 (carryingMeta=false) jenkins-hbase4.apache.org,36323,1689652533838/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@779ad6dd[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:55:42,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,39019,1689652533864 already deleted, retry=false 2023-07-18 03:55:42,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,39019,1689652533864 on jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,524 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,39019,1689652533864, splitWal=true, meta=true 2023-07-18 03:55:42,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,39019,1689652533864 (carryingMeta=true) jenkins-hbase4.apache.org,39019,1689652533864/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@38083964[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:55:42,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45597,1689652533808 already deleted, retry=false 2023-07-18 03:55:42,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45597,1689652533808 on jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45597,1689652533808, splitWal=true, meta=false 2023-07-18 03:55:42,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,45597,1689652533808 (carryingMeta=false) jenkins-hbase4.apache.org,45597,1689652533808/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@19852c86[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:55:42,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,33473,1689652533891 already deleted, retry=false 2023-07-18 03:55:42,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33473,1689652533891 on jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33473,1689652533891, splitWal=true, meta=false 2023-07-18 03:55:42,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,33473,1689652533891 (carryingMeta=false) jenkins-hbase4.apache.org,33473,1689652533891/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@791e5c12[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:55:42,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 03:55:42,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 03:55:42,539 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-18 03:55:42,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 03:55:42,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-18 03:55:42,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-18 03:55:42,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-18 03:55:42,542 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-18 03:55:42,543 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:42,543 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:55:42,543 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:55:42,544 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40805,1689652542268, sessionid=0x1017691f55f001c, setting cluster-up flag (Was=false) 2023-07-18 03:55:42,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-18 03:55:42,555 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-18 03:55:42,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:55:42,560 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.hbase-snapshot/.tmp 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:42,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,570 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689652572566 2023-07-18 03:55:42,571 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 03:55:42,571 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 03:55:42,571 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 03:55:42,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 03:55:42,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 03:55:42,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 03:55:42,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 03:55:42,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 03:55:42,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 03:55:42,577 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33473,1689652533891; numProcessing=1 2023-07-18 03:55:42,577 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39019,1689652533864; numProcessing=2 2023-07-18 03:55:42,577 DEBUG [PEWorker-5] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45597,1689652533808; numProcessing=3 2023-07-18 03:55:42,578 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36323,1689652533838; numProcessing=4 2023-07-18 03:55:42,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 03:55:42,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 03:55:42,578 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33473,1689652533891, splitWal=true, meta=false 2023-07-18 03:55:42,578 INFO [PEWorker-5] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45597,1689652533808, splitWal=true, meta=false 2023-07-18 03:55:42,578 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36323,1689652533838, splitWal=true, meta=false 2023-07-18 03:55:42,578 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652542578,5,FailOnTimeoutGroup] 2023-07-18 03:55:42,578 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39019,1689652533864, splitWal=true, meta=true 2023-07-18 03:55:42,579 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39019,1689652533864, splitWal=true, meta=true, isMeta: true 2023-07-18 03:55:42,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652542578,5,FailOnTimeoutGroup] 2023-07-18 03:55:42,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 03:55:42,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689652542583, completionTime=-1 2023-07-18 03:55:42,584 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-18 03:55:42,584 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 03:55:42,585 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864-splitting 2023-07-18 03:55:42,586 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864-splitting dir is empty, no logs to split. 2023-07-18 03:55:42,586 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,39019,1689652533864 WAL count=0, meta=true 2023-07-18 03:55:42,590 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864-splitting dir is empty, no logs to split. 2023-07-18 03:55:42,590 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,39019,1689652533864 WAL count=0, meta=true 2023-07-18 03:55:42,590 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,39019,1689652533864 WAL splitting is done? wals=0, meta=true 2023-07-18 03:55:42,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 03:55:42,593 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 03:55:42,594 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:55:42,660 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(951): ClusterId : 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:42,660 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:55:42,662 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:55:42,662 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:55:42,664 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:55:42,666 DEBUG [RS:0;jenkins-hbase4:40667] zookeeper.ReadOnlyZKClient(139): Connect 0x00ffb5a9 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:42,674 DEBUG [RS:0;jenkins-hbase4:40667] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e385bc6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:42,675 DEBUG [RS:0;jenkins-hbase4:40667] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a07ded9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:55:42,687 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40667 2023-07-18 03:55:42,687 INFO [RS:0;jenkins-hbase4:40667] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:55:42,687 INFO [RS:0;jenkins-hbase4:40667] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:55:42,687 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:55:42,688 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40805,1689652542268 with isa=jenkins-hbase4.apache.org/172.31.14.131:40667, startcode=1689652542338 2023-07-18 03:55:42,688 DEBUG [RS:0;jenkins-hbase4:40667] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:55:42,690 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54397, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:55:42,692 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40805] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,693 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:55:42,693 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36001 2023-07-18 03:55:42,693 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:55:42,694 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:55:42,695 DEBUG [RS:0;jenkins-hbase4:40667] zookeeper.ZKUtil(162): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,696 WARN [RS:0;jenkins-hbase4:40667] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:55:42,696 INFO [RS:0;jenkins-hbase4:40667] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:42,696 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,696 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40667,1689652542338] 2023-07-18 03:55:42,701 DEBUG [RS:0;jenkins-hbase4:40667] zookeeper.ZKUtil(162): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,702 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:42,702 INFO [RS:0;jenkins-hbase4:40667] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:55:42,704 INFO [RS:0;jenkins-hbase4:40667] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:55:42,705 INFO [RS:0;jenkins-hbase4:40667] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:55:42,705 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,705 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:55:42,707 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,707 DEBUG [RS:0;jenkins-hbase4:40667] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:55:42,709 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,709 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,709 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,725 INFO [RS:0;jenkins-hbase4:40667] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:55:42,726 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40667,1689652542338-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:42,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=150ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 03:55:42,739 INFO [RS:0;jenkins-hbase4:40667] regionserver.Replication(203): jenkins-hbase4.apache.org,40667,1689652542338 started 2023-07-18 03:55:42,739 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40667,1689652542338, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40667, sessionid=0x1017691f55f001d 2023-07-18 03:55:42,739 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:55:42,739 DEBUG [RS:0;jenkins-hbase4:40667] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,739 DEBUG [RS:0;jenkins-hbase4:40667] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40667,1689652542338' 2023-07-18 03:55:42,739 DEBUG [RS:0;jenkins-hbase4:40667] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40667,1689652542338' 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 03:55:42,740 DEBUG [RS:0;jenkins-hbase4:40667] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 03:55:42,741 DEBUG [RS:0;jenkins-hbase4:40667] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:55:42,741 INFO [RS:0;jenkins-hbase4:40667] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:55:42,741 INFO [RS:0;jenkins-hbase4:40667] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:55:42,744 DEBUG [jenkins-hbase4:40805] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 03:55:42,745 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40667,1689652542338, state=OPENING 2023-07-18 03:55:42,747 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 03:55:42,747 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 03:55:42,747 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40667,1689652542338}] 2023-07-18 03:55:42,843 INFO [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40667%2C1689652542338, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:42,859 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:55:42,859 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:42,859 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:55:42,861 INFO [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338/jenkins-hbase4.apache.org%2C40667%2C1689652542338.1689652542843 2023-07-18 03:55:42,861 DEBUG [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK], DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK], DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK]] 2023-07-18 03:55:42,902 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:42,902 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:42,903 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41774, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:42,907 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 03:55:42,907 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:42,909 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40667%2C1689652542338.meta, suffix=.meta, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:55:42,923 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:55:42,924 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:55:42,924 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:55:42,926 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338/jenkins-hbase4.apache.org%2C40667%2C1689652542338.meta.1689652542909.meta 2023-07-18 03:55:42,926 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK], DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK]] 2023-07-18 03:55:42,927 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:42,927 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 03:55:42,927 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 03:55:42,928 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 03:55:42,928 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 03:55:42,928 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:42,928 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 03:55:42,928 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 03:55:42,929 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:55:42,930 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:55:42,930 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:55:42,931 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:55:42,937 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/aa1e6b16427d4dbfacbeab6bd7ed6ed4 2023-07-18 03:55:42,937 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:42,938 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:55:42,939 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:42,939 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:55:42,939 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:55:42,940 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:42,940 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:55:42,941 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:55:42,941 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:55:42,941 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:55:42,947 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table/e35dec1443364b0288f9553a65a94860 2023-07-18 03:55:42,947 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:42,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:55:42,949 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:55:42,951 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:55:42,953 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:55:42,953 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68072857, jitterRate=0.014364615082740784}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:55:42,953 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:55:42,954 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689652542901 2023-07-18 03:55:42,959 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 03:55:42,959 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 03:55:42,959 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40667,1689652542338, state=OPEN 2023-07-18 03:55:42,961 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 03:55:42,961 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 03:55:42,963 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-18 03:55:42,963 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40667,1689652542338 in 214 msec 2023-07-18 03:55:42,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=16 2023-07-18 03:55:42,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 372 msec 2023-07-18 03:55:44,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1652ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-18 03:55:45,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3155ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-18 03:55:47,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-18 03:55:47,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 03:55:47,095 DEBUG [hconnection-0x29d23178-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:47,096 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41790, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:47,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6596a12435c5c3b7add032ab29aea733, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45597,1689652533808, regionLocation=jenkins-hbase4.apache.org,45597,1689652533808, openSeqNum=2 2023-07-18 03:55:47,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6d104435467e1000121c02223fd064d0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39019,1689652533864, regionLocation=jenkins-hbase4.apache.org,39019,1689652533864, openSeqNum=2 2023-07-18 03:55:47,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33473,1689652533891, regionLocation=jenkins-hbase4.apache.org,33473,1689652533891, openSeqNum=2 2023-07-18 03:55:47,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-18 03:55:47,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689652607106 2023-07-18 03:55:47,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689652667106 2023-07-18 03:55:47,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-07-18 03:55:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40805,1689652542268-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40805,1689652542268-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40805,1689652542268-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40805, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 03:55:47,130 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,36323,1689652533838 had 0 regions 2023-07-18 03:55:47,130 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45597,1689652533808 had 1 regions 2023-07-18 03:55:47,130 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33473,1689652533891 had 1 regions 2023-07-18 03:55:47,130 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,39019,1689652533864 had 2 regions 2023-07-18 03:55:47,131 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. is NOT online; state={6596a12435c5c3b7add032ab29aea733 state=OPEN, ts=1689652547106, server=jenkins-hbase4.apache.org,45597,1689652533808}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-18 03:55:47,132 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36323,1689652533838, splitWal=true, meta=false, isMeta: false 2023-07-18 03:55:47,132 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45597,1689652533808, splitWal=true, meta=false, isMeta: false 2023-07-18 03:55:47,132 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33473,1689652533891, splitWal=true, meta=false, isMeta: false 2023-07-18 03:55:47,132 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39019,1689652533864, splitWal=true, meta=true, isMeta: false 2023-07-18 03:55:47,134 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838-splitting 2023-07-18 03:55:47,136 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,45597,1689652533808/hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., unknown_server=jenkins-hbase4.apache.org,39019,1689652533864/t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0., unknown_server=jenkins-hbase4.apache.org,33473,1689652533891/t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:47,137 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,137 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,36323,1689652533838 WAL count=0, meta=false 2023-07-18 03:55:47,138 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808-splitting 2023-07-18 03:55:47,139 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,139 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,45597,1689652533808 WAL count=0, meta=false 2023-07-18 03:55:47,139 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891-splitting 2023-07-18 03:55:47,140 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,140 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,33473,1689652533891 WAL count=0, meta=false 2023-07-18 03:55:47,142 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,142 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,39019,1689652533864 WAL count=0, meta=false 2023-07-18 03:55:47,142 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,142 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,36323,1689652533838 WAL count=0, meta=false 2023-07-18 03:55:47,142 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36323,1689652533838 WAL splitting is done? wals=0, meta=false 2023-07-18 03:55:47,143 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,143 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,45597,1689652533808 WAL count=0, meta=false 2023-07-18 03:55:47,143 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45597,1689652533808 WAL splitting is done? wals=0, meta=false 2023-07-18 03:55:47,145 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,145 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,33473,1689652533891 WAL count=0, meta=false 2023-07-18 03:55:47,145 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33473,1689652533891 WAL splitting is done? wals=0, meta=false 2023-07-18 03:55:47,146 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,39019,1689652533864-splitting dir is empty, no logs to split. 2023-07-18 03:55:47,146 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,39019,1689652533864 WAL count=0, meta=false 2023-07-18 03:55:47,146 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,39019,1689652533864 WAL splitting is done? wals=0, meta=false 2023-07-18 03:55:47,146 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,36323,1689652533838 failed, ignore...File hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,36323,1689652533838-splitting does not exist. 2023-07-18 03:55:47,146 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45597,1689652533808 failed, ignore...File hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45597,1689652533808-splitting does not exist. 2023-07-18 03:55:47,148 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN}] 2023-07-18 03:55:47,148 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN}] 2023-07-18 03:55:47,149 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,33473,1689652533891 failed, ignore...File hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,33473,1689652533891-splitting does not exist. 2023-07-18 03:55:47,149 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,36323,1689652533838 after splitting done 2023-07-18 03:55:47,149 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,36323,1689652533838 from processing; numProcessing=3 2023-07-18 03:55:47,149 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN 2023-07-18 03:55:47,150 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN 2023-07-18 03:55:47,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN}] 2023-07-18 03:55:47,151 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:55:47,151 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:55:47,152 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36323,1689652533838, splitWal=true, meta=false in 4.6350 sec 2023-07-18 03:55:47,153 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN 2023-07-18 03:55:47,153 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:55:47,167 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:55:47,301 DEBUG [jenkins-hbase4:40805] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 03:55:47,304 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6596a12435c5c3b7add032ab29aea733, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:47,304 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:47,304 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=6d104435467e1000121c02223fd064d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:47,304 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652547304"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652547304"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652547304"}]},"ts":"1689652547304"} 2023-07-18 03:55:47,304 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652547304"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652547304"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652547304"}]},"ts":"1689652547304"} 2023-07-18 03:55:47,304 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652547304"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652547304"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652547304"}]},"ts":"1689652547304"} 2023-07-18 03:55:47,307 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure 6596a12435c5c3b7add032ab29aea733, server=jenkins-hbase4.apache.org,40667,1689652542338}] 2023-07-18 03:55:47,308 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 6d104435467e1000121c02223fd064d0, server=jenkins-hbase4.apache.org,40667,1689652542338}] 2023-07-18 03:55:47,310 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure d59bee2bde49e7e5abb27771a5bfd3f4, server=jenkins-hbase4.apache.org,40667,1689652542338}] 2023-07-18 03:55:47,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:47,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6d104435467e1000121c02223fd064d0, NAME => 't1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:47,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:47,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,466 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,466 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:47,466 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6596a12435c5c3b7add032ab29aea733, NAME => 'hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:47,467 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,467 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:47,467 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,467 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,467 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1 2023-07-18 03:55:47,467 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1 2023-07-18 03:55:47,467 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d104435467e1000121c02223fd064d0 columnFamilyName cf1 2023-07-18 03:55:47,468 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,469 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info 2023-07-18 03:55:47,469 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info 2023-07-18 03:55:47,470 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6596a12435c5c3b7add032ab29aea733 columnFamilyName info 2023-07-18 03:55:47,476 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/f7b510ff27dc4e3286f94f9b58c2ef73 2023-07-18 03:55:47,480 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(310): Store=6d104435467e1000121c02223fd064d0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:47,480 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,481 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/norep 2023-07-18 03:55:47,481 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/norep 2023-07-18 03:55:47,481 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d104435467e1000121c02223fd064d0 columnFamilyName norep 2023-07-18 03:55:47,482 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(310): Store=6d104435467e1000121c02223fd064d0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:47,482 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info/7d27b66e1d264872bc50ba77246226b1 2023-07-18 03:55:47,483 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(310): Store=6596a12435c5c3b7add032ab29aea733/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:47,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,485 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:55:47,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6d104435467e1000121c02223fd064d0 2023-07-18 03:55:47,488 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:55:47,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6d104435467e1000121c02223fd064d0; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70620719, jitterRate=0.05233071744441986}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:55:47,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:55:47,489 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6596a12435c5c3b7add032ab29aea733; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67445496, jitterRate=0.005016207695007324}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:55:47,489 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:55:47,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0., pid=25, masterSystemTime=1689652547460 2023-07-18 03:55:47,490 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., pid=24, masterSystemTime=1689652547460 2023-07-18 03:55:47,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:47,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:55:47,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:47,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d59bee2bde49e7e5abb27771a5bfd3f4, NAME => 't2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:55:47,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,493 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=6d104435467e1000121c02223fd064d0, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:47,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:55:47,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:47,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,494 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652547493"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652547493"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652547493"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652547493"}]},"ts":"1689652547493"} 2023-07-18 03:55:47,494 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:55:47,494 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6596a12435c5c3b7add032ab29aea733, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:47,494 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652547494"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652547494"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652547494"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652547494"}]},"ts":"1689652547494"} 2023-07-18 03:55:47,495 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,497 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1 2023-07-18 03:55:47,497 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1 2023-07-18 03:55:47,497 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d59bee2bde49e7e5abb27771a5bfd3f4 columnFamilyName cf1 2023-07-18 03:55:47,498 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-18 03:55:47,498 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 6d104435467e1000121c02223fd064d0, server=jenkins-hbase4.apache.org,40667,1689652542338 in 188 msec 2023-07-18 03:55:47,498 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-18 03:55:47,498 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure 6596a12435c5c3b7add032ab29aea733, server=jenkins-hbase4.apache.org,40667,1689652542338 in 189 msec 2023-07-18 03:55:47,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=16 2023-07-18 03:55:47,500 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,39019,1689652533864 after splitting done 2023-07-18 03:55:47,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN in 350 msec 2023-07-18 03:55:47,500 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,39019,1689652533864 from processing; numProcessing=2 2023-07-18 03:55:47,500 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=17 2023-07-18 03:55:47,500 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45597,1689652533808 after splitting done 2023-07-18 03:55:47,500 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN in 350 msec 2023-07-18 03:55:47,500 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45597,1689652533808 from processing; numProcessing=1 2023-07-18 03:55:47,501 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,39019,1689652533864, splitWal=true, meta=true in 4.9760 sec 2023-07-18 03:55:47,502 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45597,1689652533808, splitWal=true, meta=false in 4.9740 sec 2023-07-18 03:55:47,505 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/3d270ab02d934440a4e65c4aa122741c 2023-07-18 03:55:47,505 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(310): Store=d59bee2bde49e7e5abb27771a5bfd3f4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:47,505 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,506 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/norep 2023-07-18 03:55:47,506 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/norep 2023-07-18 03:55:47,507 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d59bee2bde49e7e5abb27771a5bfd3f4 columnFamilyName norep 2023-07-18 03:55:47,507 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(310): Store=d59bee2bde49e7e5abb27771a5bfd3f4/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:55:47,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:55:47,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:55:47,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d59bee2bde49e7e5abb27771a5bfd3f4; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67815677, jitterRate=0.010532334446907043}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:55:47,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:55:47,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4., pid=26, masterSystemTime=1689652547460 2023-07-18 03:55:47,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:47,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:55:47,517 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:55:47,517 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652547517"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652547517"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652547517"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652547517"}]},"ts":"1689652547517"} 2023-07-18 03:55:47,520 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-18 03:55:47,520 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure d59bee2bde49e7e5abb27771a5bfd3f4, server=jenkins-hbase4.apache.org,40667,1689652542338 in 209 msec 2023-07-18 03:55:47,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=18 2023-07-18 03:55:47,522 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33473,1689652533891 after splitting done 2023-07-18 03:55:47,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN in 370 msec 2023-07-18 03:55:47,522 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33473,1689652533891 from processing; numProcessing=0 2023-07-18 03:55:47,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33473,1689652533891, splitWal=true, meta=false in 4.9940 sec 2023-07-18 03:55:47,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:48,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-18 03:55:48,146 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-18 03:55:48,154 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-18 03:55:48,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.787sec 2023-07-18 03:55:48,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 03:55:48,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 03:55:48,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 03:55:48,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40805,1689652542268-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 03:55:48,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40805,1689652542268-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 03:55:48,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 03:55:48,164 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x255702b4 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:48,168 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21ba383f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:48,169 DEBUG [hconnection-0x4102fd6e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:48,172 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41792, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:48,174 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-18 03:55:48,174 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x255702b4 to 127.0.0.1:60939 2023-07-18 03:55:48,175 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:55:48,176 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:40805 after: jenkins-hbase4.apache.org:40805 2023-07-18 03:55:48,176 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x2d47f211 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:48,181 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b78835a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:48,683 DEBUG [hconnection-0x26e4efd4-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:48,685 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41796, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:55:48,703 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-18 03:55:48,704 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-18 03:55:48,704 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-18 03:55:48,704 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-18 03:55:48,710 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689652548710 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:48,714 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication16896525487100x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:48,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689652548710-0x1017691f55f0022 connected Start Replication Server start 2023-07-18 03:55:48,718 INFO [Listener at localhost/46383] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:55:48,731 INFO [Listener at localhost/46383] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 03:55:48,733 DEBUG [Listener at localhost/46383] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:55:48,743 INFO [Listener at localhost/46383] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:55:48,744 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,42337,1689652529544, jenkins-hbase4.apache.org,36729,1689652529501] other RSs: [] 2023-07-18 03:55:48,763 DEBUG [Listener at localhost/46383.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x7f8e8f4c to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:48,768 DEBUG [Listener at localhost/46383.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@56cead46, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:48,768 INFO [Listener at localhost/46383.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:48,772 DEBUG [Listener at localhost/46383.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:48,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f0024 connected 2023-07-18 03:55:48,773 INFO [Listener at localhost/46383.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:52,071 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42337,1689652529544/1's WALs to 1689652548718.syncuptool.replication.org,1234,1 2023-07-18 03:55:52,077 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 with data PBUF\x08\xF8\xA9\x01 2023-07-18 03:55:52,080 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,42337,1689652529544/1's WALs to 1689652548718.syncuptool.replication.org,1234,1 2023-07-18 03:55:52,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:52,967 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:55:54,082 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42337,1689652529544] 2023-07-18 03:55:54,082 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42337,1689652529544] 2023-07-18 03:55:54,097 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42337,1689652529544] 2023-07-18 03:55:54,097 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,42337,1689652529544, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:55:54,097 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:54,118 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544] zookeeper.ReadOnlyZKClient(139): Connect 0x460e10e6 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:54,123 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@52302996, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:54,124 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:54,128 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:54,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f0026 connected 2023-07-18 03:55:54,129 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,42337,1689652529544 (queues=1) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:54,129 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42337%2C1689652529544 2023-07-18 03:55:54,133 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,42337,1689652529544, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 03:55:54,134 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42337%2C1689652529544 2023-07-18 03:55:54,141 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992 2023-07-18 03:55:54,141 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 03:55:54,141 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 03:55:54,142 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42337,1689652529544 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/f95930cf45f370dfce61fdb9733fd578/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-18 03:55:54,145 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:54,146 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:54,148 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40667] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:54,148 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40667] zookeeper.ReadOnlyZKClient(139): Connect 0x7fc701da to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:54,155 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40667] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7100f052, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:54,157 DEBUG [hconnection-0x753f6aaa-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:55:54,158 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51784, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-18 03:55:54,171 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40667] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:54,171 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,42337,1689652529544] 2023-07-18 03:55:54,173 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42337%2C1689652529544.1689652530992] 2023-07-18 03:55:54,174 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,42337,1689652529544 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 03:55:54,174 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C42337%2C1689652529544 of peer 1-jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:54,275 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:54,277 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652548710-0x1017691f55f0022, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689652548718.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,42337,1689652529544 2023-07-18 03:55:54,278 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544.replicationSource.shipperjenkins-hbase4.apache.org%2C42337%2C1689652529544,1-jenkins-hbase4.apache.org,42337,1689652529544] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,42337,1689652529544 with the following stats: Total replicated edits: 1, current progress: 2023-07-18 03:55:57,325 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,36729,1689652529501/1's WALs to 1689652548718.syncuptool.replication.org,1234,1 2023-07-18 03:55:57,328 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992 with data PBUF\x08\xE4Y 2023-07-18 03:55:57,330 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,36729,1689652529501/1's WALs to 1689652548718.syncuptool.replication.org,1234,1 2023-07-18 03:55:57,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:55:57,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 1 2023-07-18 03:55:59,332 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36729,1689652529501] 2023-07-18 03:55:59,332 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36729,1689652529501] 2023-07-18 03:55:59,343 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36729,1689652529501] 2023-07-18 03:55:59,343 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,36729,1689652529501, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:55:59,343 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:59,363 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501] zookeeper.ReadOnlyZKClient(139): Connect 0x5b6d6fe7 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:55:59,367 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@31d597d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:55:59,367 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:55:59,369 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:55:59,371 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f0029 connected 2023-07-18 03:55:59,371 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,36729,1689652529501 (queues=1) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:55:59,371 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C36729%2C1689652529501 2023-07-18 03:55:59,372 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,36729,1689652529501, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 03:55:59,373 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C36729%2C1689652529501 2023-07-18 03:55:59,380 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992 2023-07-18 03:55:59,380 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 03:55:59,380 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 03:55:59,381 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,36729,1689652529501 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992, lastWalPosition=16520, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-18 03:55:59,384 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:55:59,385 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51798, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:55:59,386 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40667] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:55:59,393 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40667] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:55:59,393 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,36729,1689652529501] 2023-07-18 03:55:59,396 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C36729%2C1689652529501.1689652530992] 2023-07-18 03:55:59,397 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,36729,1689652529501 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 03:55:59,397 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C36729%2C1689652529501 of peer 1-jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:59,497 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:59,500 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652548710-0x1017691f55f0022, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689652548718.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,36729,1689652529501 2023-07-18 03:55:59,500 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501.replicationSource.shipperjenkins-hbase4.apache.org%2C36729%2C1689652529501,1-jenkins-hbase4.apache.org,36729,1689652529501] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,36729,1689652529501 with the following stats: Total replicated edits: 1, current progress: 2023-07-18 03:55:59,778 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:56:02,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:56:02,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-18 03:56:05,000 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:56:07,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:56:07,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-18 03:56:08,746 INFO [Listener at localhost/46383] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 03:56:08,850 DEBUG [Listener at localhost/46383.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0024, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:08,850 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f8e8f4c to 127.0.0.1:60939 2023-07-18 03:56:08,851 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:09,054 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652548710-0x1017691f55f0022, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:09,054 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652548710-0x1017691f55f0022, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:09,067 INFO [Listener at localhost/46383] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-18 03:56:09,067 DEBUG [Listener at localhost/46383] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-18 03:56:09,069 INFO [Listener at localhost/46383] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 03:56:09,069 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:09,069 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:09,069 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:56:09,070 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:09,070 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:56:09,070 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:56:09,070 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44325 2023-07-18 03:56:09,071 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:09,072 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:09,073 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44325 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:09,075 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:443250x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:09,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44325-0x1017691f55f002a connected 2023-07-18 03:56:09,079 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:56:09,079 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:56:09,080 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 03:56:09,080 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44325 2023-07-18 03:56:09,080 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44325 2023-07-18 03:56:09,080 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44325 2023-07-18 03:56:09,085 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44325 2023-07-18 03:56:09,085 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44325 2023-07-18 03:56:09,085 INFO [Listener at localhost/46383] master.HMaster(444): hbase.rootdir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26, hbase.cluster.distributed=false 2023-07-18 03:56:09,086 DEBUG [pool-541-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:56:09,097 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:56:09,099 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40321 2023-07-18 03:56:09,100 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:56:09,104 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:56:09,104 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:09,105 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:09,106 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40321 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:09,112 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:403210x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:09,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40321-0x1017691f55f002b connected 2023-07-18 03:56:09,115 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:403210x0, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:56:09,115 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:56:09,116 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 03:56:09,119 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40321 2023-07-18 03:56:09,119 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40321 2023-07-18 03:56:09,121 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40321 2023-07-18 03:56:09,123 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40321 2023-07-18 03:56:09,123 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40321 2023-07-18 03:56:09,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,126 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 03:56:09,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,129 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 03:56:09,129 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 03:56:09,129 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:56:09,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 03:56:09,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,44325,1689652569068 from backup master directory 2023-07-18 03:56:09,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 03:56:09,133 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,133 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 03:56:09,133 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:56:09,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:09,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6b7fa359 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:09,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65abd241, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:09,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:56:09,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 03:56:09,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:09,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089 to hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089-dead as it is dead 2023-07-18 03:56:09,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089-dead/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 2023-07-18 03:56:09,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089-dead/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 after 0ms 2023-07-18 03:56:09,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089-dead/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 to hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 2023-07-18 03:56:09,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,41639,1689652528089-dead 2023-07-18 03:56:09,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44325%2C1689652569068, suffix=, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,44325,1689652569068, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/oldWALs, maxLogs=10 2023-07-18 03:56:09,214 DEBUG [RS-EventLoopGroup-15-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:56:09,214 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:56:09,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/WALs/jenkins-hbase4.apache.org,44325,1689652569068/jenkins-hbase4.apache.org%2C44325%2C1689652569068.1689652569199 2023-07-18 03:56:09,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK], DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK]] 2023-07-18 03:56:09,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:09,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:09,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:09,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:09,221 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:09,222 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 03:56:09,223 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 03:56:09,232 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a594b5d4016e4d97b0d35b24053aa8e6 2023-07-18 03:56:09,232 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:09,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-18 03:56:09,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 2023-07-18 03:56:09,238 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 2023-07-18 03:56:09,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41639%2C1689652528089.1689652529996 2023-07-18 03:56:09,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:09,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-18 03:56:09,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66686199, jitterRate=-0.00629819929599762}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:56:09,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:56:09,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 03:56:09,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 03:56:09,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 03:56:09,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 03:56:09,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 03:56:09,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-18 03:56:09,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-18 03:56:09,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-18 03:56:09,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-18 03:56:09,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-18 03:56:09,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-18 03:56:09,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-18 03:56:09,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-18 03:56:09,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 03:56:09,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-18 03:56:09,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36729,1689652529501, table=hbase:meta, region=1588230740 2023-07-18 03:56:09,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-18 03:56:09,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42337,1689652529544 already deleted, retry=false 2023-07-18 03:56:09,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42337,1689652529544 on jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42337,1689652529544, splitWal=true, meta=false 2023-07-18 03:56:09,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,42337,1689652529544 (carryingMeta=false) jenkins-hbase4.apache.org,42337,1689652529544/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@194a9fc6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:56:09,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,36729,1689652529501 already deleted, retry=false 2023-07-18 03:56:09,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,36729,1689652529501 on jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,36729,1689652529501, splitWal=true, meta=true 2023-07-18 03:56:09,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,36729,1689652529501 (carryingMeta=true) jenkins-hbase4.apache.org,36729,1689652529501/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@77807b45[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:56:09,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 03:56:09,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 03:56:09,273 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-18 03:56:09,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 03:56:09,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-18 03:56:09,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-18 03:56:09,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-18 03:56:09,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-18 03:56:09,279 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 03:56:09,279 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 03:56:09,279 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:56:09,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44325,1689652569068, sessionid=0x1017691f55f002a, setting cluster-up flag (Was=false) 2023-07-18 03:56:09,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-18 03:56:09,287 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-18 03:56:09,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:09,290 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/.hbase-snapshot/.tmp 2023-07-18 03:56:09,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:09,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:09,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:09,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:09,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 03:56:09,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:56:09,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689652599295 2023-07-18 03:56:09,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 03:56:09,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 03:56:09,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 03:56:09,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 03:56:09,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 03:56:09,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 03:56:09,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,297 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36729,1689652529501; numProcessing=1 2023-07-18 03:56:09,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 03:56:09,298 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36729,1689652529501, splitWal=true, meta=true 2023-07-18 03:56:09,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 03:56:09,298 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42337,1689652529544; numProcessing=2 2023-07-18 03:56:09,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 03:56:09,298 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42337,1689652529544, splitWal=true, meta=false 2023-07-18 03:56:09,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 03:56:09,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 03:56:09,299 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36729,1689652529501, splitWal=true, meta=true, isMeta: true 2023-07-18 03:56:09,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652569298,5,FailOnTimeoutGroup] 2023-07-18 03:56:09,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652569303,5,FailOnTimeoutGroup] 2023-07-18 03:56:09,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 03:56:09,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689652569303, completionTime=-1 2023-07-18 03:56:09,303 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-18 03:56:09,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 03:56:09,304 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501-splitting 2023-07-18 03:56:09,305 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501-splitting dir is empty, no logs to split. 2023-07-18 03:56:09,305 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,36729,1689652529501 WAL count=0, meta=true 2023-07-18 03:56:09,307 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501-splitting dir is empty, no logs to split. 2023-07-18 03:56:09,307 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,36729,1689652529501 WAL count=0, meta=true 2023-07-18 03:56:09,307 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36729,1689652529501 WAL splitting is done? wals=0, meta=true 2023-07-18 03:56:09,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 03:56:09,309 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 03:56:09,310 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:09,324 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(951): ClusterId : 0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 2023-07-18 03:56:09,324 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:56:09,326 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:56:09,326 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:56:09,328 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:56:09,329 DEBUG [RS:0;jenkins-hbase4:40321] zookeeper.ReadOnlyZKClient(139): Connect 0x074bcd1c to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:09,334 DEBUG [RS:0;jenkins-hbase4:40321] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@98c2385, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:09,334 DEBUG [RS:0;jenkins-hbase4:40321] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@db3a66c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:56:09,342 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40321 2023-07-18 03:56:09,342 INFO [RS:0;jenkins-hbase4:40321] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:56:09,342 INFO [RS:0;jenkins-hbase4:40321] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:56:09,342 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:56:09,343 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44325,1689652569068 with isa=jenkins-hbase4.apache.org/172.31.14.131:40321, startcode=1689652569096 2023-07-18 03:56:09,343 DEBUG [RS:0;jenkins-hbase4:40321] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:56:09,344 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35015, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:56:09,345 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44325] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,346 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26 2023-07-18 03:56:09,346 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40155 2023-07-18 03:56:09,346 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:56:09,347 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:56:09,347 DEBUG [RS:0;jenkins-hbase4:40321] zookeeper.ZKUtil(162): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,347 WARN [RS:0;jenkins-hbase4:40321] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:56:09,348 INFO [RS:0;jenkins-hbase4:40321] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:09,348 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:09,348 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=51ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 03:56:09,363 INFO [RS:0;jenkins-hbase4:40321] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 03:56:09,363 DEBUG [RS:0;jenkins-hbase4:40321] zookeeper.ZKUtil(162): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,364 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:56:09,364 INFO [RS:0;jenkins-hbase4:40321] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:56:09,366 INFO [RS:0;jenkins-hbase4:40321] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:56:09,366 INFO [RS:0;jenkins-hbase4:40321] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:56:09,366 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,366 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:56:09,368 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,368 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,368 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,368 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,368 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,368 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,369 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:56:09,369 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,369 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,369 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,369 DEBUG [RS:0;jenkins-hbase4:40321] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:09,370 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,370 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,370 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,382 INFO [RS:0;jenkins-hbase4:40321] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:56:09,382 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40321,1689652569096-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:09,393 INFO [RS:0;jenkins-hbase4:40321] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:56:09,394 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689652548718.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:09,404 INFO [RS:0;jenkins-hbase4:40321] regionserver.Replication(203): jenkins-hbase4.apache.org,40321,1689652569096 started 2023-07-18 03:56:09,404 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40321,1689652569096, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40321, sessionid=0x1017691f55f002b 2023-07-18 03:56:09,404 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40321,1689652569096' 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40321,1689652569096' 2023-07-18 03:56:09,405 DEBUG [RS:0;jenkins-hbase4:40321] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-18 03:56:09,406 DEBUG [RS:0;jenkins-hbase4:40321] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-18 03:56:09,406 DEBUG [RS:0;jenkins-hbase4:40321] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:56:09,406 INFO [RS:0;jenkins-hbase4:40321] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:56:09,406 INFO [RS:0;jenkins-hbase4:40321] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:56:09,415 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x4ab4c094 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:09,419 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6b3a7de1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:09,419 INFO [RS:0;jenkins-hbase4:40321.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:09,423 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:09,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f002f connected 2023-07-18 03:56:09,427 INFO [RS:0;jenkins-hbase4:40321.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:56:09,460 DEBUG [jenkins-hbase4:44325] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 03:56:09,462 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40321,1689652569096, state=OPENING 2023-07-18 03:56:09,464 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-18 03:56:09,464 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 03:56:09,464 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40321,1689652569096}] 2023-07-18 03:56:09,508 INFO [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40321%2C1689652569096, suffix=, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs, maxLogs=10 2023-07-18 03:56:09,526 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:56:09,526 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:56:09,536 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C40321%2C1689652569096 for peer 1 2023-07-18 03:56:09,536 INFO [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096/jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509 2023-07-18 03:56:09,536 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C40321%2C1689652569096 2023-07-18 03:56:09,537 INFO [RS:0;jenkins-hbase4:40321] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 03:56:09,537 DEBUG [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK], DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK]] 2023-07-18 03:56:09,537 INFO [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C40321%2C1689652569096 2023-07-18 03:56:09,541 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:09,623 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:09,623 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:56:09,625 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50548, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:56:09,628 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 03:56:09,628 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:09,630 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40321%2C1689652569096.meta, suffix=.meta, logDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096, archiveDir=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs, maxLogs=10 2023-07-18 03:56:09,644 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK] 2023-07-18 03:56:09,645 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK] 2023-07-18 03:56:09,650 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:09,654 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096/jenkins-hbase4.apache.org%2C40321%2C1689652569096.meta.1689652569630.meta 2023-07-18 03:56:09,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42603,DS-5dcf1289-5fbd-4710-b303-15a85102257f,DISK], DatanodeInfoWithStorage[127.0.0.1:38001,DS-b2fdeefa-b7e0-4ac3-ac87-2be6a0926235,DISK]] 2023-07-18 03:56:09,654 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:09,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 03:56:09,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 03:56:09,655 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 03:56:09,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 03:56:09,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:09,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 03:56:09,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 03:56:09,657 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:56:09,658 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info 2023-07-18 03:56:09,658 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info 2023-07-18 03:56:09,658 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:56:09,670 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info/51c9bb8f378b4fb4abb67c058897fc76 2023-07-18 03:56:09,670 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:09,670 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:56:09,671 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:56:09,672 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:56:09,672 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:56:09,678 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier/d8c0ee8d8ad841798f467b6bfed755f0 2023-07-18 03:56:09,678 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:09,679 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:56:09,680 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table 2023-07-18 03:56:09,680 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table 2023-07-18 03:56:09,680 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:56:09,687 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/table/f40694dec6b74e12ae0468b21b6767ee 2023-07-18 03:56:09,688 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:09,689 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740 2023-07-18 03:56:09,690 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740 2023-07-18 03:56:09,693 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:56:09,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:56:09,696 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74739235, jitterRate=0.11370138823986053}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:56:09,696 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:56:09,696 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689652569623 2023-07-18 03:56:09,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 03:56:09,702 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 03:56:09,702 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40321,1689652569096, state=OPEN 2023-07-18 03:56:09,704 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-18 03:56:09,704 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 03:56:09,708 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-18 03:56:09,709 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40321,1689652569096 in 242 msec 2023-07-18 03:56:09,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-18 03:56:09,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 401 msec 2023-07-18 03:56:09,755 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:09,860 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:09,965 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,071 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,177 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,283 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,388 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,494 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,599 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,704 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,808 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:10,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1554ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-18 03:56:10,913 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,017 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,122 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,227 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,336 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,443 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,548 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,653 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,757 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,862 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:11,966 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,071 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,176 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,280 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-18 03:56:12,385 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,490 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,594 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,699 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:56:12,739 INFO [jenkins-hbase4:40667Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-18 03:56:12,804 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:12,909 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,013 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,118 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,223 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,328 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,432 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,536 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,641 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,745 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-18 03:56:13,811 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 03:56:13,815 DEBUG [hconnection-0x6b1b408d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:13,816 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:56:13,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=12eb339fe9277c226d5f3d9d08310c02, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36729,1689652529501, regionLocation=jenkins-hbase4.apache.org,36729,1689652529501, openSeqNum=2 2023-07-18 03:56:13,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f1f3df17ddf51de240de231140850f62, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36729,1689652529501, regionLocation=jenkins-hbase4.apache.org,36729,1689652529501, openSeqNum=2 2023-07-18 03:56:13,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f95930cf45f370dfce61fdb9733fd578, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42337,1689652529544, regionLocation=jenkins-hbase4.apache.org,42337,1689652529544, openSeqNum=2 2023-07-18 03:56:13,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-18 03:56:13,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689652633823 2023-07-18 03:56:13,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689652693823 2023-07-18 03:56:13,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-18 03:56:13,839 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,36729,1689652529501 had 3 regions 2023-07-18 03:56:13,840 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42337,1689652529544 had 1 regions 2023-07-18 03:56:13,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44325,1689652569068-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:13,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44325,1689652569068-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:13,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44325,1689652569068-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:13,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44325, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:13,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:13,840 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. is NOT online; state={12eb339fe9277c226d5f3d9d08310c02 state=OPEN, ts=1689652573823, server=jenkins-hbase4.apache.org,36729,1689652529501}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-18 03:56:13,841 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36729,1689652529501, splitWal=true, meta=true, isMeta: false 2023-07-18 03:56:13,841 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42337,1689652529544, splitWal=true, meta=false, isMeta: false 2023-07-18 03:56:13,844 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,36729,1689652529501/hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02., unknown_server=jenkins-hbase4.apache.org,36729,1689652529501/t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62., unknown_server=jenkins-hbase4.apache.org,42337,1689652529544/t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:13,844 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501-splitting dir is empty, no logs to split. 2023-07-18 03:56:13,844 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,36729,1689652529501 WAL count=0, meta=false 2023-07-18 03:56:13,845 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544-splitting 2023-07-18 03:56:13,853 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544-splitting dir is empty, no logs to split. 2023-07-18 03:56:13,853 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,42337,1689652529544 WAL count=0, meta=false 2023-07-18 03:56:13,855 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,36729,1689652529501-splitting dir is empty, no logs to split. 2023-07-18 03:56:13,855 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,36729,1689652529501 WAL count=0, meta=false 2023-07-18 03:56:13,855 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36729,1689652529501 WAL splitting is done? wals=0, meta=false 2023-07-18 03:56:13,857 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN}, {pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN}] 2023-07-18 03:56:13,857 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544-splitting dir is empty, no logs to split. 2023-07-18 03:56:13,857 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,42337,1689652529544 WAL count=0, meta=false 2023-07-18 03:56:13,857 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42337,1689652529544 WAL splitting is done? wals=0, meta=false 2023-07-18 03:56:13,858 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:13,858 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN 2023-07-18 03:56:13,858 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN 2023-07-18 03:56:13,858 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:13,859 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:13,860 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,42337,1689652529544 failed, ignore...File hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,42337,1689652529544-splitting does not exist. 2023-07-18 03:56:13,861 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN}] 2023-07-18 03:56:13,862 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN 2023-07-18 03:56:13,862 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:13,962 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:14,009 DEBUG [jenkins-hbase4:44325] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 03:56:14,012 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=12eb339fe9277c226d5f3d9d08310c02, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:14,012 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=f95930cf45f370dfce61fdb9733fd578, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:14,012 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=f1f3df17ddf51de240de231140850f62, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:14,012 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652574012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652574012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652574012"}]},"ts":"1689652574012"} 2023-07-18 03:56:14,012 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652574012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652574012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652574012"}]},"ts":"1689652574012"} 2023-07-18 03:56:14,012 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652574012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652574012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652574012"}]},"ts":"1689652574012"} 2023-07-18 03:56:14,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=23, state=RUNNABLE; OpenRegionProcedure 12eb339fe9277c226d5f3d9d08310c02, server=jenkins-hbase4.apache.org,40321,1689652569096}] 2023-07-18 03:56:14,016 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=24, state=RUNNABLE; OpenRegionProcedure f95930cf45f370dfce61fdb9733fd578, server=jenkins-hbase4.apache.org,40321,1689652569096}] 2023-07-18 03:56:14,016 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=22, state=RUNNABLE; OpenRegionProcedure f1f3df17ddf51de240de231140850f62, server=jenkins-hbase4.apache.org,40321,1689652569096}] 2023-07-18 03:56:14,068 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:14,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:14,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1f3df17ddf51de240de231140850f62, NAME => 't1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:14,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:14,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,173 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,174 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 03:56:14,174 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:14,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 12eb339fe9277c226d5f3d9d08310c02, NAME => 'hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:14,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:14,174 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1 2023-07-18 03:56:14,174 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1 2023-07-18 03:56:14,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,175 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1f3df17ddf51de240de231140850f62 columnFamilyName cf1 2023-07-18 03:56:14,176 INFO [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,177 DEBUG [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info 2023-07-18 03:56:14,177 DEBUG [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info 2023-07-18 03:56:14,178 INFO [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12eb339fe9277c226d5f3d9d08310c02 columnFamilyName info 2023-07-18 03:56:14,181 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36e6fc5b788b41c19d8786bebda95cb2 2023-07-18 03:56:14,181 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1/36e6fc5b788b41c19d8786bebda95cb2 2023-07-18 03:56:14,181 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(310): Store=f1f3df17ddf51de240de231140850f62/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:14,181 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,182 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep 2023-07-18 03:56:14,182 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep 2023-07-18 03:56:14,183 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1f3df17ddf51de240de231140850f62 columnFamilyName norep 2023-07-18 03:56:14,193 DEBUG [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/info/3759eb675381438895a87e71f03c0574 2023-07-18 03:56:14,193 INFO [StoreOpener-12eb339fe9277c226d5f3d9d08310c02-1] regionserver.HStore(310): Store=12eb339fe9277c226d5f3d9d08310c02/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:14,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,196 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9f0cf58662854a048aad2485229f1a4f 2023-07-18 03:56:14,196 DEBUG [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep/9f0cf58662854a048aad2485229f1a4f 2023-07-18 03:56:14,196 INFO [StoreOpener-f1f3df17ddf51de240de231140850f62-1] regionserver.HStore(310): Store=f1f3df17ddf51de240de231140850f62/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:14,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:56:14,204 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:14,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:14,205 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 12eb339fe9277c226d5f3d9d08310c02; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68068383, jitterRate=0.01429794728755951}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:56:14,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 12eb339fe9277c226d5f3d9d08310c02: 2023-07-18 03:56:14,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1f3df17ddf51de240de231140850f62; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60754526, jitterRate=-0.09468701481819153}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:56:14,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1f3df17ddf51de240de231140850f62: 2023-07-18 03:56:14,206 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02., pid=25, masterSystemTime=1689652574167 2023-07-18 03:56:14,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62., pid=27, masterSystemTime=1689652574167 2023-07-18 03:56:14,208 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:14,208 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:14,209 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=12eb339fe9277c226d5f3d9d08310c02, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:14,209 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652574209"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652574209"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652574209"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652574209"}]},"ts":"1689652574209"} 2023-07-18 03:56:14,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:14,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:14,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:14,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f95930cf45f370dfce61fdb9733fd578, NAME => 't2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:14,210 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=f1f3df17ddf51de240de231140850f62, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:14,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,210 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652574210"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652574210"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652574210"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652574210"}]},"ts":"1689652574210"} 2023-07-18 03:56:14,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:14,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,212 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,213 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=23 2023-07-18 03:56:14,213 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=23, state=SUCCESS; OpenRegionProcedure 12eb339fe9277c226d5f3d9d08310c02, server=jenkins-hbase4.apache.org,40321,1689652569096 in 196 msec 2023-07-18 03:56:14,213 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1 2023-07-18 03:56:14,213 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1 2023-07-18 03:56:14,213 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f95930cf45f370dfce61fdb9733fd578 columnFamilyName cf1 2023-07-18 03:56:14,214 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=22 2023-07-18 03:56:14,214 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=22, state=SUCCESS; OpenRegionProcedure f1f3df17ddf51de240de231140850f62, server=jenkins-hbase4.apache.org,40321,1689652569096 in 196 msec 2023-07-18 03:56:14,214 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=12eb339fe9277c226d5f3d9d08310c02, ASSIGN in 357 msec 2023-07-18 03:56:14,216 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=19 2023-07-18 03:56:14,216 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,36729,1689652529501 after splitting done 2023-07-18 03:56:14,216 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=f1f3df17ddf51de240de231140850f62, ASSIGN in 358 msec 2023-07-18 03:56:14,216 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,36729,1689652529501 from processing; numProcessing=1 2023-07-18 03:56:14,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36729,1689652529501, splitWal=true, meta=true in 4.9540 sec 2023-07-18 03:56:14,221 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 100e4311365a4b829c861a164a543574 2023-07-18 03:56:14,221 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1/100e4311365a4b829c861a164a543574 2023-07-18 03:56:14,221 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(310): Store=f95930cf45f370dfce61fdb9733fd578/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:14,221 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,222 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep 2023-07-18 03:56:14,222 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep 2023-07-18 03:56:14,222 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f95930cf45f370dfce61fdb9733fd578 columnFamilyName norep 2023-07-18 03:56:14,228 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f2be17687e34c4f9789b573b25ee02d 2023-07-18 03:56:14,229 DEBUG [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(539): loaded hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep/4f2be17687e34c4f9789b573b25ee02d 2023-07-18 03:56:14,229 INFO [StoreOpener-f95930cf45f370dfce61fdb9733fd578-1] regionserver.HStore(310): Store=f95930cf45f370dfce61fdb9733fd578/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:14,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:56:14,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:14,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f95930cf45f370dfce61fdb9733fd578; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63324297, jitterRate=-0.05639444291591644}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:56:14,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f95930cf45f370dfce61fdb9733fd578: 2023-07-18 03:56:14,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578., pid=26, masterSystemTime=1689652574167 2023-07-18 03:56:14,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:14,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:14,237 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=f95930cf45f370dfce61fdb9733fd578, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:14,237 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652574237"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652574237"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652574237"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652574237"}]},"ts":"1689652574237"} 2023-07-18 03:56:14,240 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=24 2023-07-18 03:56:14,240 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=24, state=SUCCESS; OpenRegionProcedure f95930cf45f370dfce61fdb9733fd578, server=jenkins-hbase4.apache.org,40321,1689652569096 in 223 msec 2023-07-18 03:56:14,241 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=18 2023-07-18 03:56:14,242 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=f95930cf45f370dfce61fdb9733fd578, ASSIGN in 379 msec 2023-07-18 03:56:14,242 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42337,1689652529544 after splitting done 2023-07-18 03:56:14,242 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42337,1689652529544 from processing; numProcessing=0 2023-07-18 03:56:14,243 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42337,1689652529544, splitWal=true, meta=false in 4.9830 sec 2023-07-18 03:56:14,279 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,279 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 03:56:14,279 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,279 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096/jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509, lastWalPosition=1162, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 03:56:14,356 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:56:14,389 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,404 INFO [jenkins-hbase4:40321Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C40321%2C1689652569096]: currently replicating from: hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096/jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509 at position: 1162 2023-07-18 03:56:14,494 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,599 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,703 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,807 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-18 03:56:14,855 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-18 03:56:14,857 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-18 03:56:14,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.724sec 2023-07-18 03:56:14,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 03:56:14,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 03:56:14,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 03:56:14,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44325,1689652569068-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 03:56:14,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44325,1689652569068-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 03:56:14,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 03:56:14,913 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:14,931 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x3228d4bc to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:14,936 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@759ee5d7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:14,937 DEBUG [hconnection-0x24aa4178-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:14,939 INFO [RS-EventLoopGroup-15-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57794, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:56:14,945 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-18 03:56:14,945 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3228d4bc to 127.0.0.1:60939 2023-07-18 03:56:14,945 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:14,946 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:44325 after: jenkins-hbase4.apache.org:44325 2023-07-18 03:56:14,947 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x4352b8f2 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:14,953 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35058061, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:14,953 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d47f211 to 127.0.0.1:60939 2023-07-18 03:56:14,953 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:14,953 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 03:56:14,953 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(257): Found active master hash=552088731, stopped=false 2023-07-18 03:56:14,953 INFO [Listener at localhost/46383] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:56:14,961 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:56:14,961 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:56:14,961 INFO [Listener at localhost/46383] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 03:56:14,961 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:56:14,963 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7ac44eb7 to 127.0.0.1:60939 2023-07-18 03:56:14,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:56:14,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:56:14,963 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:14,963 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40667,1689652542338' ***** 2023-07-18 03:56:14,963 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:56:14,964 INFO [RS:0;jenkins-hbase4:40667] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:56:14,964 INFO [RS:0;jenkins-hbase4:40667] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:56:14,964 INFO [RS:0;jenkins-hbase4:40667] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:56:14,964 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:56:14,964 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(3305): Received CLOSE for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:14,965 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(3305): Received CLOSE for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:14,965 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(3305): Received CLOSE for 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:14,965 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:56:14,965 DEBUG [RS:0;jenkins-hbase4:40667] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x00ffb5a9 to 127.0.0.1:60939 2023-07-18 03:56:14,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6596a12435c5c3b7add032ab29aea733, disabling compactions & flushes 2023-07-18 03:56:14,966 DEBUG [RS:0;jenkins-hbase4:40667] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:14,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:14,966 INFO [RS:0;jenkins-hbase4:40667] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:56:14,966 INFO [RS:0;jenkins-hbase4:40667] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:56:14,966 INFO [RS:0;jenkins-hbase4:40667] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:56:14,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:14,966 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 03:56:14,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. after waiting 0 ms 2023-07-18 03:56:14,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:14,966 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 03:56:14,966 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 6596a12435c5c3b7add032ab29aea733=hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., d59bee2bde49e7e5abb27771a5bfd3f4=t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4., 6d104435467e1000121c02223fd064d0=t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.} 2023-07-18 03:56:14,967 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:56:14,967 DEBUG [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1504): Waiting on 1588230740, 6596a12435c5c3b7add032ab29aea733, 6d104435467e1000121c02223fd064d0, d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:14,967 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:56:14,967 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:56:14,967 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:56:14,968 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:56:14,969 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-18 03:56:14,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-18 03:56:14,975 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:14,975 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:56:14,975 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:14,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d59bee2bde49e7e5abb27771a5bfd3f4, disabling compactions & flushes 2023-07-18 03:56:14,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:14,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:14,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. after waiting 0 ms 2023-07-18 03:56:14,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:14,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d59bee2bde49e7e5abb27771a5bfd3f4 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-18 03:56:14,986 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/info/d4135d8464eb4a0da8b9cfd3f8f7c7da 2023-07-18 03:56:14,992 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/info/d4135d8464eb4a0da8b9cfd3f8f7c7da as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/d4135d8464eb4a0da8b9cfd3f8f7c7da 2023-07-18 03:56:14,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/.tmp/cf1/bbc414ea1b3449fd9d09d2d92f427eb0 2023-07-18 03:56:14,998 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/d4135d8464eb4a0da8b9cfd3f8f7c7da, entries=24, sequenceid=31, filesize=7.4 K 2023-07-18 03:56:14,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbc414ea1b3449fd9d09d2d92f427eb0 2023-07-18 03:56:14,999 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 30ms, sequenceid=31, compaction requested=false 2023-07-18 03:56:15,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/.tmp/cf1/bbc414ea1b3449fd9d09d2d92f427eb0 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/bbc414ea1b3449fd9d09d2d92f427eb0 2023-07-18 03:56:15,010 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-18 03:56:15,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 03:56:15,011 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:56:15,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:56:15,012 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 03:56:15,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbc414ea1b3449fd9d09d2d92f427eb0 2023-07-18 03:56:15,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/bbc414ea1b3449fd9d09d2d92f427eb0, entries=100, sequenceid=14, filesize=7.9 K 2023-07-18 03:56:15,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for d59bee2bde49e7e5abb27771a5bfd3f4 in 37ms, sequenceid=14, compaction requested=false 2023-07-18 03:56:15,022 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:15,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-18 03:56:15,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:15,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:56:15,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:15,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6d104435467e1000121c02223fd064d0, disabling compactions & flushes 2023-07-18 03:56:15,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:15,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:15,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. after waiting 0 ms 2023-07-18 03:56:15,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:15,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6d104435467e1000121c02223fd064d0 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-18 03:56:15,033 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:56:15,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/.tmp/cf1/967b742e58634ba0879b99eae13ac0ae 2023-07-18 03:56:15,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 967b742e58634ba0879b99eae13ac0ae 2023-07-18 03:56:15,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/.tmp/cf1/967b742e58634ba0879b99eae13ac0ae as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/967b742e58634ba0879b99eae13ac0ae 2023-07-18 03:56:15,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 967b742e58634ba0879b99eae13ac0ae 2023-07-18 03:56:15,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/967b742e58634ba0879b99eae13ac0ae, entries=50, sequenceid=14, filesize=6.4 K 2023-07-18 03:56:15,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 6d104435467e1000121c02223fd064d0 in 26ms, sequenceid=14, compaction requested=false 2023-07-18 03:56:15,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-18 03:56:15,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:15,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:56:15,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:15,128 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:15,167 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40667,1689652542338; all regions closed. 2023-07-18 03:56:15,174 DEBUG [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:56:15,174 INFO [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40667%2C1689652542338.meta:.meta(num 1689652542909) 2023-07-18 03:56:15,180 DEBUG [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:56:15,180 INFO [RS:0;jenkins-hbase4:40667] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40667%2C1689652542338:(num 1689652542843) 2023-07-18 03:56:15,180 DEBUG [RS:0;jenkins-hbase4:40667] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:15,180 INFO [RS:0;jenkins-hbase4:40667] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:56:15,180 INFO [RS:0;jenkins-hbase4:40667] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 03:56:15,180 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:56:15,181 DEBUG [RS:0;jenkins-hbase4:40667] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7fc701da to 127.0.0.1:60939 2023-07-18 03:56:15,181 DEBUG [RS:0;jenkins-hbase4:40667] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:15,182 INFO [RS:0;jenkins-hbase4:40667] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40667 2023-07-18 03:56:15,184 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,40667,1689652542338 2023-07-18 03:56:15,185 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0026, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:15,185 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:15,185 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:15,185 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0029, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:15,185 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:15,185 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:15,185 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40667,1689652542338] 2023-07-18 03:56:15,185 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40667,1689652542338; numProcessing=1 2023-07-18 03:56:15,187 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40667,1689652542338 already deleted, retry=false 2023-07-18 03:56:15,187 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40667,1689652542338 expired; onlineServers=0 2023-07-18 03:56:15,187 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40805,1689652542268' ***** 2023-07-18 03:56:15,187 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 03:56:15,187 DEBUG [M:0;jenkins-hbase4:40805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24d7a45e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:56:15,188 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40805,1689652542268 2023-07-18 03:56:15,188 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40805,1689652542268; all regions closed. 2023-07-18 03:56:15,188 DEBUG [M:0;jenkins-hbase4:40805] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:15,188 DEBUG [M:0;jenkins-hbase4:40805] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 03:56:15,188 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 03:56:15,188 DEBUG [M:0;jenkins-hbase4:40805] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 03:56:15,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652542578] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652542578,5,FailOnTimeoutGroup] 2023-07-18 03:56:15,189 INFO [M:0;jenkins-hbase4:40805] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 03:56:15,189 INFO [M:0;jenkins-hbase4:40805] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 03:56:15,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652542578] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652542578,5,FailOnTimeoutGroup] 2023-07-18 03:56:15,189 INFO [M:0;jenkins-hbase4:40805] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 03:56:15,190 DEBUG [M:0;jenkins-hbase4:40805] master.HMaster(1512): Stopping service threads 2023-07-18 03:56:15,190 INFO [M:0;jenkins-hbase4:40805] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 03:56:15,191 ERROR [M:0;jenkins-hbase4:40805] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-18 03:56:15,191 INFO [M:0;jenkins-hbase4:40805] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 03:56:15,191 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-18 03:56:15,191 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 03:56:15,191 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:56:15,191 DEBUG [M:0;jenkins-hbase4:40805] zookeeper.ZKUtil(398): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-18 03:56:15,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:56:15,191 WARN [M:0;jenkins-hbase4:40805] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 03:56:15,191 INFO [M:0;jenkins-hbase4:40805] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 03:56:15,192 INFO [M:0;jenkins-hbase4:40805] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 03:56:15,193 DEBUG [M:0;jenkins-hbase4:40805] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:56:15,193 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:15,193 DEBUG [M:0;jenkins-hbase4:40805] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:15,193 DEBUG [M:0;jenkins-hbase4:40805] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:56:15,193 DEBUG [M:0;jenkins-hbase4:40805] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:15,193 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.70 KB heapSize=47.16 KB 2023-07-18 03:56:15,209 INFO [M:0;jenkins-hbase4:40805] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.70 KB at sequenceid=239 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2924ff7288524c3886a94e1d7b04dff9 2023-07-18 03:56:15,214 INFO [M:0;jenkins-hbase4:40805] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2924ff7288524c3886a94e1d7b04dff9 2023-07-18 03:56:15,215 DEBUG [M:0;jenkins-hbase4:40805] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2924ff7288524c3886a94e1d7b04dff9 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2924ff7288524c3886a94e1d7b04dff9 2023-07-18 03:56:15,219 INFO [M:0;jenkins-hbase4:40805] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2924ff7288524c3886a94e1d7b04dff9 2023-07-18 03:56:15,219 INFO [M:0;jenkins-hbase4:40805] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2924ff7288524c3886a94e1d7b04dff9, entries=20, sequenceid=239, filesize=5.6 K 2023-07-18 03:56:15,220 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegion(2948): Finished flush of dataSize ~36.70 KB/37578, heapSize ~47.15 KB/48280, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=239, compaction requested=false 2023-07-18 03:56:15,223 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:15,223 DEBUG [M:0;jenkins-hbase4:40805] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:56:15,227 INFO [M:0;jenkins-hbase4:40805] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 03:56:15,227 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:56:15,228 INFO [M:0;jenkins-hbase4:40805] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40805 2023-07-18 03:56:15,233 DEBUG [M:0;jenkins-hbase4:40805] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,40805,1689652542268 already deleted, retry=false 2023-07-18 03:56:15,235 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:15,339 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:15,444 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:15,450 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:15,450 INFO [M:0;jenkins-hbase4:40805] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40805,1689652542268; zookeeper connection closed. 2023-07-18 03:56:15,451 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40805-0x1017691f55f001c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:15,548 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-18 03:56:15,551 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:15,551 INFO [RS:0;jenkins-hbase4:40667] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40667,1689652542338; zookeeper connection closed. 2023-07-18 03:56:15,551 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40667-0x1017691f55f001d, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:15,552 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@60eda96e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@60eda96e 2023-07-18 03:56:15,552 INFO [Listener at localhost/46383] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-18 03:56:15,554 DEBUG [hconnection-0x4ff506b4-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:15,555 INFO [RS-EventLoopGroup-15-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57806, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:56:15,659 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 11726 2023-07-18 03:56:15,659 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceWALReader(162): Read 100 WAL entries eligible for replication 2023-07-18 03:56:15,660 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/112=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/113=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/114=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/115=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/116=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/117=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/118=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/119=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/120=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/121=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/122=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/123=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/124=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/125=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/126=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/127=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/128=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/129=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/130=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/131=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/132=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/133=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/134=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/135=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/136=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/137=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/138=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/139=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/140=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/141=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/142=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/143=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/144=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/145=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/146=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/147=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/148=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/149=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/150=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/151=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/152=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/153=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/154=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/155=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/156=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/157=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/158=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/159=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/160=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/161=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/162=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/163=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/164=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/165=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/166=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/167=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/168=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/169=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/170=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/171=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/172=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/173=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/174=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/175=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/176=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/177=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/178=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/179=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/180=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/181=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/182=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/183=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/184=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/185=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/186=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/187=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/188=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/189=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/190=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/191=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/192=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/193=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/194=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/195=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/196=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/197=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/198=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/199=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/200=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/201=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/202=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/203=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/204=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/205=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/206=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/207=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/208=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/209=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/210=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/211=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/WALs/jenkins-hbase4.apache.org,40321,1689652569096/jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509, lastWalPosition=11726, nbRowKeys=100, nbHFiles=0, heapSize=20800, lastSeqIds={}, endOfFile=false,usedBufferSize=20800] 2023-07-18 03:56:15,660 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 11726 2023-07-18 03:56:15,663 WARN [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:56:15,764 WARN [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:56:15,770 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 25658 2023-07-18 03:56:15,771 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceWALReader(162): Read 130 WAL entries eligible for replication 2023-07-18 03:56:15,771 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 25658 2023-07-18 03:56:15,837 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4352b8f2 to 127.0.0.1:60939 2023-07-18 03:56:15,837 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:15,837 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 03:56:15,837 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(257): Found active master hash=202620997, stopped=false 2023-07-18 03:56:15,837 INFO [Listener at localhost/46383] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:15,839 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 03:56:15,839 INFO [Listener at localhost/46383] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 03:56:15,839 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 03:56:15,840 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:56:15,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:56:15,842 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b7fa359 to 127.0.0.1:60939 2023-07-18 03:56:15,843 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:15,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 03:56:15,843 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40321,1689652569096' ***** 2023-07-18 03:56:15,843 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:56:15,843 INFO [RS:0;jenkins-hbase4:40321] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:56:15,843 INFO [RS:0;jenkins-hbase4:40321] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:56:15,843 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:56:15,843 INFO [RS:0;jenkins-hbase4:40321] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:56:15,844 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(3305): Received CLOSE for f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:15,844 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(3305): Received CLOSE for f1f3df17ddf51de240de231140850f62 2023-07-18 03:56:15,845 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(3305): Received CLOSE for 12eb339fe9277c226d5f3d9d08310c02 2023-07-18 03:56:15,845 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:15,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f95930cf45f370dfce61fdb9733fd578, disabling compactions & flushes 2023-07-18 03:56:15,845 DEBUG [RS:0;jenkins-hbase4:40321] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x074bcd1c to 127.0.0.1:60939 2023-07-18 03:56:15,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:15,845 DEBUG [RS:0;jenkins-hbase4:40321] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:15,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:15,846 INFO [RS:0;jenkins-hbase4:40321] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:56:15,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. after waiting 0 ms 2023-07-18 03:56:15,846 INFO [RS:0;jenkins-hbase4:40321] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:56:15,846 INFO [RS:0;jenkins-hbase4:40321] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:56:15,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:15,846 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 03:56:15,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f95930cf45f370dfce61fdb9733fd578 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-18 03:56:15,846 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 03:56:15,846 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1478): Online Regions={f95930cf45f370dfce61fdb9733fd578=t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578., f1f3df17ddf51de240de231140850f62=t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62., 1588230740=hbase:meta,,1.1588230740, 12eb339fe9277c226d5f3d9d08310c02=hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02.} 2023-07-18 03:56:15,846 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1504): Waiting on 12eb339fe9277c226d5f3d9d08310c02, 1588230740, f1f3df17ddf51de240de231140850f62, f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:15,847 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:56:15,847 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:56:15,847 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:56:15,847 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:56:15,847 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:56:15,847 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-18 03:56:15,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/cf1/816e5029f79c413dbeda454f557bcec5 2023-07-18 03:56:15,867 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/info/e0a56defe0874180b49a86b63a8425ea 2023-07-18 03:56:15,876 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:56:15,886 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] wal.ProtobufLogReader(420): EOF at position 33573 2023-07-18 03:56:15,886 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSourceWALReader(162): Read 70 WAL entries eligible for replication 2023-07-18 03:56:15,916 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/rep_barrier/fa05ab2dd7cf42e19eb35d7c8897c147 2023-07-18 03:56:15,931 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/info/e0a56defe0874180b49a86b63a8425ea as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info/e0a56defe0874180b49a86b63a8425ea 2023-07-18 03:56:15,941 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/info/e0a56defe0874180b49a86b63a8425ea, entries=24, sequenceid=31, filesize=7.4 K 2023-07-18 03:56:15,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/.tmp/rep_barrier/fa05ab2dd7cf42e19eb35d7c8897c147 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier/fa05ab2dd7cf42e19eb35d7c8897c147 2023-07-18 03:56:15,948 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/rep_barrier/fa05ab2dd7cf42e19eb35d7c8897c147, entries=2, sequenceid=31, filesize=4.9 K 2023-07-18 03:56:15,949 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 102ms, sequenceid=31, compaction requested=false 2023-07-18 03:56:15,964 WARN [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:56:15,967 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-18 03:56:15,967 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 03:56:15,968 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:56:15,968 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:56:15,968 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 03:56:16,046 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1504): Waiting on 12eb339fe9277c226d5f3d9d08310c02, f1f3df17ddf51de240de231140850f62, f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:16,247 DEBUG [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1504): Waiting on 12eb339fe9277c226d5f3d9d08310c02, f1f3df17ddf51de240de231140850f62, f95930cf45f370dfce61fdb9733fd578 2023-07-18 03:56:16,265 WARN [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 03:56:16,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/norep/91eb9a2496ac4904a33ebe2a5d02b5a8 2023-07-18 03:56:16,324 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/cf1/816e5029f79c413dbeda454f557bcec5 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1/816e5029f79c413dbeda454f557bcec5 2023-07-18 03:56:16,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/cf1/816e5029f79c413dbeda454f557bcec5, entries=200, sequenceid=413, filesize=12.5 K 2023-07-18 03:56:16,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/.tmp/norep/91eb9a2496ac4904a33ebe2a5d02b5a8 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep/91eb9a2496ac4904a33ebe2a5d02b5a8 2023-07-18 03:56:16,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/norep/91eb9a2496ac4904a33ebe2a5d02b5a8, entries=1, sequenceid=413, filesize=4.8 K 2023-07-18 03:56:16,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for f95930cf45f370dfce61fdb9733fd578 in 497ms, sequenceid=413, compaction requested=false 2023-07-18 03:56:16,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t2_syncup/f95930cf45f370dfce61fdb9733fd578/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-18 03:56:16,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:16,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f95930cf45f370dfce61fdb9733fd578: 2023-07-18 03:56:16,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689652536335.f95930cf45f370dfce61fdb9733fd578. 2023-07-18 03:56:16,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1f3df17ddf51de240de231140850f62, disabling compactions & flushes 2023-07-18 03:56:16,355 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:16,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:16,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. after waiting 0 ms 2023-07-18 03:56:16,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:16,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f1f3df17ddf51de240de231140850f62 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-18 03:56:16,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/cf1/ae85485e9b0b48498cd7cf5c87cb80f5 2023-07-18 03:56:16,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/norep/52de3fd7bca341b3b24462e4c0c058ee 2023-07-18 03:56:16,385 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/cf1/ae85485e9b0b48498cd7cf5c87cb80f5 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1/ae85485e9b0b48498cd7cf5c87cb80f5 2023-07-18 03:56:16,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/cf1/ae85485e9b0b48498cd7cf5c87cb80f5, entries=100, sequenceid=213, filesize=8.4 K 2023-07-18 03:56:16,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/.tmp/norep/52de3fd7bca341b3b24462e4c0c058ee as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep/52de3fd7bca341b3b24462e4c0c058ee 2023-07-18 03:56:16,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/norep/52de3fd7bca341b3b24462e4c0c058ee, entries=1, sequenceid=213, filesize=4.8 K 2023-07-18 03:56:16,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for f1f3df17ddf51de240de231140850f62 in 41ms, sequenceid=213, compaction requested=false 2023-07-18 03:56:16,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/default/t1_syncup/f1f3df17ddf51de240de231140850f62/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-18 03:56:16,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:16,404 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-18 03:56:16,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1f3df17ddf51de240de231140850f62: 2023-07-18 03:56:16,404 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-18 03:56:16,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689652535678.f1f3df17ddf51de240de231140850f62. 2023-07-18 03:56:16,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 12eb339fe9277c226d5f3d9d08310c02, disabling compactions & flushes 2023-07-18 03:56:16,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:16,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:16,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. after waiting 0 ms 2023-07-18 03:56:16,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:16,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/data/hbase/namespace/12eb339fe9277c226d5f3d9d08310c02/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-18 03:56:16,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:16,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 12eb339fe9277c226d5f3d9d08310c02: 2023-07-18 03:56:16,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689652531467.12eb339fe9277c226d5f3d9d08310c02. 2023-07-18 03:56:16,447 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40321,1689652569096; all regions closed. 2023-07-18 03:56:16,454 DEBUG [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs 2023-07-18 03:56:16,454 INFO [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40321%2C1689652569096.meta:.meta(num 1689652569630) 2023-07-18 03:56:16,460 DEBUG [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs 2023-07-18 03:56:16,460 INFO [RS:0;jenkins-hbase4:40321] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40321%2C1689652569096:(num 1689652569509) 2023-07-18 03:56:16,460 DEBUG [RS:0;jenkins-hbase4:40321] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:16,460 INFO [RS:0;jenkins-hbase4:40321] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:56:16,460 INFO [RS:0;jenkins-hbase4:40321] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 03:56:16,460 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:56:16,460 INFO [RS:0;jenkins-hbase4:40321] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 03:56:16,562 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f002f, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:16,562 DEBUG [RS:0;jenkins-hbase4:40321] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ab4c094 to 127.0.0.1:60939 2023-07-18 03:56:16,562 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f002f, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:16,563 DEBUG [RS:0;jenkins-hbase4:40321] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:16,664 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.HBaseInterClusterReplicationEndpoint(243): Waiting for peers [Source for peer 1]: Interrupted while sleeping between retries 2023-07-18 03:56:16,665 WARN [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-18 03:56:16,665 DEBUG [RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1] regionserver.HBaseInterClusterReplicationEndpoint(243): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-07-18 03:56:16,665 INFO [RS:0;jenkins-hbase4:40321] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:40321.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1 terminated 2023-07-18 03:56:16,666 INFO [RS:0;jenkins-hbase4:40321] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40321 2023-07-18 03:56:16,671 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:16,671 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:56:16,672 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 03:56:16,673 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:16,673 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40321,1689652569096; numProcessing=1 2023-07-18 03:56:16,674 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,40321,1689652569096 already deleted, retry=false 2023-07-18 03:56:16,674 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40321,1689652569096 expired; onlineServers=0 2023-07-18 03:56:16,674 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44325,1689652569068' ***** 2023-07-18 03:56:16,674 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 03:56:16,675 DEBUG [M:0;jenkins-hbase4:44325] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@82c551f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:56:16,675 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44325,1689652569068 2023-07-18 03:56:16,675 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44325,1689652569068; all regions closed. 2023-07-18 03:56:16,675 DEBUG [M:0;jenkins-hbase4:44325] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:16,675 DEBUG [M:0;jenkins-hbase4:44325] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 03:56:16,675 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 03:56:16,675 DEBUG [M:0;jenkins-hbase4:44325] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 03:56:16,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652569298] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652569298,5,FailOnTimeoutGroup] 2023-07-18 03:56:16,676 INFO [M:0;jenkins-hbase4:44325] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 03:56:16,676 INFO [M:0;jenkins-hbase4:44325] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 03:56:16,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652569303] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652569303,5,FailOnTimeoutGroup] 2023-07-18 03:56:16,677 INFO [M:0;jenkins-hbase4:44325] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 03:56:16,677 DEBUG [M:0;jenkins-hbase4:44325] master.HMaster(1512): Stopping service threads 2023-07-18 03:56:16,677 INFO [M:0;jenkins-hbase4:44325] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 03:56:16,677 ERROR [M:0;jenkins-hbase4:44325] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-18 03:56:16,677 INFO [M:0;jenkins-hbase4:44325] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 03:56:16,678 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 03:56:16,678 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-18 03:56:16,678 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 03:56:16,678 DEBUG [M:0;jenkins-hbase4:44325] zookeeper.ZKUtil(398): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-18 03:56:16,678 WARN [M:0;jenkins-hbase4:44325] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 03:56:16,678 INFO [M:0;jenkins-hbase4:44325] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 03:56:16,678 INFO [M:0;jenkins-hbase4:44325] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 03:56:16,679 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 03:56:16,680 DEBUG [M:0;jenkins-hbase4:44325] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:56:16,680 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:16,680 DEBUG [M:0;jenkins-hbase4:44325] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:16,680 DEBUG [M:0;jenkins-hbase4:44325] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:56:16,680 DEBUG [M:0;jenkins-hbase4:44325] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:16,680 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.95 KB 2023-07-18 03:56:16,690 INFO [M:0;jenkins-hbase4:44325] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/51fb2bfb92e24ef6b750df6e1c911713 2023-07-18 03:56:16,695 DEBUG [M:0;jenkins-hbase4:44325] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/51fb2bfb92e24ef6b750df6e1c911713 as hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/51fb2bfb92e24ef6b750df6e1c911713 2023-07-18 03:56:16,699 INFO [M:0;jenkins-hbase4:44325] regionserver.HStore(1080): Added hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/51fb2bfb92e24ef6b750df6e1c911713, entries=10, sequenceid=218, filesize=5.8 K 2023-07-18 03:56:16,700 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30423, heapSize ~36.94 KB/37824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=218, compaction requested=false 2023-07-18 03:56:16,701 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:16,701 DEBUG [M:0;jenkins-hbase4:44325] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:56:16,705 INFO [M:0;jenkins-hbase4:44325] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 03:56:16,705 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:56:16,705 INFO [M:0;jenkins-hbase4:44325] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44325 2023-07-18 03:56:16,708 DEBUG [M:0;jenkins-hbase4:44325] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,44325,1689652569068 already deleted, retry=false 2023-07-18 03:56:16,773 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:16,773 INFO [RS:0;jenkins-hbase4:40321] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40321,1689652569096; zookeeper connection closed. 2023-07-18 03:56:16,773 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:40321-0x1017691f55f002b, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:16,774 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4668f27c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4668f27c 2023-07-18 03:56:16,774 INFO [Listener at localhost/46383] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-18 03:56:16,873 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:16,873 INFO [M:0;jenkins-hbase4:44325] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44325,1689652569068; zookeeper connection closed. 2023-07-18 03:56:16,873 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:44325-0x1017691f55f002a, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:56:16,877 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:56:16,878 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40927 2023-07-18 03:56:16,878 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:16,880 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:16,881 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40927 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:16,883 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:409270x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:16,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40927-0x1017691f55f0032 connected 2023-07-18 03:56:16,886 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:56:16,887 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:56:16,887 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:56:16,887 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40927 2023-07-18 03:56:16,888 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40927 2023-07-18 03:56:16,888 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40927 2023-07-18 03:56:16,888 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40927 2023-07-18 03:56:16,888 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40927 2023-07-18 03:56:16,888 INFO [Listener at localhost/46383] master.HMaster(444): hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb, hbase.cluster.distributed=false 2023-07-18 03:56:16,889 DEBUG [pool-638-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 03:56:16,901 INFO [Listener at localhost/46383] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 03:56:16,902 INFO [Listener at localhost/46383] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45575 2023-07-18 03:56:16,902 INFO [Listener at localhost/46383] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 03:56:16,906 DEBUG [Listener at localhost/46383] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 03:56:16,906 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:16,908 INFO [Listener at localhost/46383] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:16,909 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45575 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:16,911 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:455750x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:16,913 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:56:16,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45575-0x1017691f55f0033 connected 2023-07-18 03:56:16,913 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:56:16,914 DEBUG [Listener at localhost/46383] zookeeper.ZKUtil(164): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 03:56:16,914 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45575 2023-07-18 03:56:16,914 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45575 2023-07-18 03:56:16,914 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45575 2023-07-18 03:56:16,915 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45575 2023-07-18 03:56:16,915 DEBUG [Listener at localhost/46383] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45575 2023-07-18 03:56:16,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:16,921 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 03:56:16,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:16,923 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:56:16,923 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:56:16,923 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 03:56:16,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 03:56:16,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,40927,1689652576876 from backup master directory 2023-07-18 03:56:16,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 03:56:16,928 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:16,928 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 03:56:16,928 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:56:16,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:16,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 03:56:17,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x43f6871b to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:17,012 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a949027, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:17,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 03:56:17,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 03:56:17,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:17,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268 to hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268-dead as it is dead 2023-07-18 03:56:17,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268-dead/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 2023-07-18 03:56:17,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268-dead/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 after 1ms 2023-07-18 03:56:17,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268-dead/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 to hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 2023-07-18 03:56:17,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40805,1689652542268-dead 2023-07-18 03:56:17,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:17,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40927%2C1689652576876, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40927,1689652576876, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/oldWALs, maxLogs=10 2023-07-18 03:56:17,040 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:56:17,040 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:56:17,040 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:56:17,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/WALs/jenkins-hbase4.apache.org,40927,1689652576876/jenkins-hbase4.apache.org%2C40927%2C1689652576876.1689652577023 2023-07-18 03:56:17,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK]] 2023-07-18 03:56:17,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:17,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:17,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:17,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:17,044 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:17,045 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 03:56:17,046 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 03:56:17,054 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2924ff7288524c3886a94e1d7b04dff9 2023-07-18 03:56:17,054 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2924ff7288524c3886a94e1d7b04dff9 2023-07-18 03:56:17,057 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e8074e048f744f8db8f92f79e670bb4b 2023-07-18 03:56:17,057 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:17,058 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-18 03:56:17,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 2023-07-18 03:56:17,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 140, firstSequenceIdInLog=120, maxSequenceIdInLog=241, path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 2023-07-18 03:56:17,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C40805%2C1689652542268.1689652542432 2023-07-18 03:56:17,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 03:56:17,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/241.seqid, newMaxSeqId=241, maxSeqId=118 2023-07-18 03:56:17,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=242; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61687371, jitterRate=-0.08078654110431671}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:56:17,068 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:56:17,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 03:56:17,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 03:56:17,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 03:56:17,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 03:56:17,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 03:56:17,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-18 03:56:17,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-18 03:56:17,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-18 03:56:17,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-18 03:56:17,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-18 03:56:17,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-18 03:56:17,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-18 03:56:17,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 03:56:17,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-18 03:56:17,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40667,1689652542338, table=hbase:meta, region=1588230740 2023-07-18 03:56:17,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-18 03:56:17,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,40667,1689652542338 already deleted, retry=false 2023-07-18 03:56:17,083 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,40667,1689652542338 on jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:17,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,40667,1689652542338, splitWal=true, meta=true 2023-07-18 03:56:17,084 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,40667,1689652542338 (carryingMeta=true) jenkins-hbase4.apache.org,40667,1689652542338/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@e8c7057[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 03:56:17,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 03:56:17,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 03:56:17,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-18 03:56:17,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 03:56:17,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-18 03:56:17,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-18 03:56:17,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-18 03:56:17,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-18 03:56:17,098 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:56:17,098 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 03:56:17,098 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:56:17,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40927,1689652576876, sessionid=0x1017691f55f0032, setting cluster-up flag (Was=false) 2023-07-18 03:56:17,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-18 03:56:17,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:17,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-18 03:56:17,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:17,109 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/.hbase-snapshot/.tmp 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:56:17,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689652607115 2023-07-18 03:56:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 03:56:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 03:56:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 03:56:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 03:56:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 03:56:17,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 03:56:17,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,119 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40667,1689652542338; numProcessing=1 2023-07-18 03:56:17,119 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40667,1689652542338, splitWal=true, meta=true 2023-07-18 03:56:17,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 03:56:17,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 03:56:17,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 03:56:17,121 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(951): ClusterId : 7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:56:17,121 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40667,1689652542338, splitWal=true, meta=true, isMeta: true 2023-07-18 03:56:17,121 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 03:56:17,123 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338-splitting 2023-07-18 03:56:17,123 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338-splitting dir is empty, no logs to split. 2023-07-18 03:56:17,123 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,40667,1689652542338 WAL count=0, meta=true 2023-07-18 03:56:17,124 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 03:56:17,124 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 03:56:17,125 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338-splitting dir is empty, no logs to split. 2023-07-18 03:56:17,126 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,40667,1689652542338 WAL count=0, meta=true 2023-07-18 03:56:17,126 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,40667,1689652542338 WAL splitting is done? wals=0, meta=true 2023-07-18 03:56:17,126 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 03:56:17,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 03:56:17,136 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 03:56:17,136 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 03:56:17,136 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 03:56:17,136 DEBUG [RS:0;jenkins-hbase4:45575] zookeeper.ReadOnlyZKClient(139): Connect 0x0ce26c93 to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:17,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652577136,5,FailOnTimeoutGroup] 2023-07-18 03:56:17,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652577137,5,FailOnTimeoutGroup] 2023-07-18 03:56:17,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 03:56:17,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,137 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:17,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689652577138, completionTime=-1 2023-07-18 03:56:17,138 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-18 03:56:17,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 03:56:17,142 DEBUG [RS:0;jenkins-hbase4:45575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@225937f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:17,142 DEBUG [RS:0;jenkins-hbase4:45575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61a07e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:56:17,150 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45575 2023-07-18 03:56:17,151 INFO [RS:0;jenkins-hbase4:45575] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 03:56:17,151 INFO [RS:0;jenkins-hbase4:45575] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 03:56:17,151 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 03:56:17,151 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40927,1689652576876 with isa=jenkins-hbase4.apache.org/172.31.14.131:45575, startcode=1689652576900 2023-07-18 03:56:17,151 DEBUG [RS:0;jenkins-hbase4:45575] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 03:56:17,153 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33621, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 03:56:17,154 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40927] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,155 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb 2023-07-18 03:56:17,155 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36001 2023-07-18 03:56:17,155 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 03:56:17,156 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0029, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:17,156 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:17,156 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:17,156 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0026, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:17,156 DEBUG [RS:0;jenkins-hbase4:45575] zookeeper.ZKUtil(162): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,156 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:17,156 WARN [RS:0;jenkins-hbase4:45575] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 03:56:17,157 INFO [RS:0;jenkins-hbase4:45575] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:17,157 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,158 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45575,1689652576900] 2023-07-18 03:56:17,163 DEBUG [RS:0;jenkins-hbase4:45575] zookeeper.ZKUtil(162): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,164 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:56:17,164 INFO [RS:0;jenkins-hbase4:45575] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 03:56:17,171 INFO [RS:0;jenkins-hbase4:45575] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 03:56:17,171 INFO [RS:0;jenkins-hbase4:45575] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 03:56:17,171 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,171 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 03:56:17,173 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,173 DEBUG [RS:0;jenkins-hbase4:45575] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 03:56:17,176 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,176 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,176 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 03:56:17,191 INFO [RS:0;jenkins-hbase4:45575] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 03:56:17,191 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45575,1689652576900-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:17,202 INFO [RS:0;jenkins-hbase4:45575] regionserver.Replication(203): jenkins-hbase4.apache.org,45575,1689652576900 started 2023-07-18 03:56:17,202 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45575,1689652576900, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45575, sessionid=0x1017691f55f0033 2023-07-18 03:56:17,202 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 03:56:17,202 DEBUG [RS:0;jenkins-hbase4:45575] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,202 DEBUG [RS:0;jenkins-hbase4:45575] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45575,1689652576900' 2023-07-18 03:56:17,202 DEBUG [RS:0;jenkins-hbase4:45575] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 03:56:17,202 DEBUG [RS:0;jenkins-hbase4:45575] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45575,1689652576900' 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 03:56:17,203 DEBUG [RS:0;jenkins-hbase4:45575] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 03:56:17,203 INFO [RS:0;jenkins-hbase4:45575] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 03:56:17,203 INFO [RS:0;jenkins-hbase4:45575] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 03:56:17,287 DEBUG [jenkins-hbase4:40927] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 03:56:17,289 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45575,1689652576900, state=OPENING 2023-07-18 03:56:17,291 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 03:56:17,291 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 03:56:17,291 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45575,1689652576900}] 2023-07-18 03:56:17,305 INFO [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45575%2C1689652576900, suffix=, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45575,1689652576900, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:56:17,320 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:56:17,320 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK] 2023-07-18 03:56:17,321 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:56:17,322 INFO [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45575,1689652576900/jenkins-hbase4.apache.org%2C45575%2C1689652576900.1689652577305 2023-07-18 03:56:17,322 DEBUG [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:45629,DS-cfef0c64-8b80-4de2-8266-7d6085198f26,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK]] 2023-07-18 03:56:17,446 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:17,446 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:56:17,447 INFO [RS-EventLoopGroup-17-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39382, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:56:17,451 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 03:56:17,451 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:17,452 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45575%2C1689652576900.meta, suffix=.meta, logDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45575,1689652576900, archiveDir=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs, maxLogs=10 2023-07-18 03:56:17,466 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK] 2023-07-18 03:56:17,466 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK] 2023-07-18 03:56:17,466 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK] 2023-07-18 03:56:17,468 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,45575,1689652576900/jenkins-hbase4.apache.org%2C45575%2C1689652576900.meta.1689652577453.meta 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34991,DS-61c2271a-7ac7-4361-8dac-57336b0fc9a7,DISK], DatanodeInfoWithStorage[127.0.0.1:34653,DS-55311c2a-76ef-4031-bb64-f570b49240d7,DISK], DatanodeInfoWithStorage[127.0.0.1:44219,DS-70babad7-9ea0-4927-991b-f59f632e2026,DISK]] 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 03:56:17,469 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 03:56:17,469 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 03:56:17,471 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 03:56:17,471 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:56:17,472 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info 2023-07-18 03:56:17,472 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 03:56:17,478 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/aa1e6b16427d4dbfacbeab6bd7ed6ed4 2023-07-18 03:56:17,482 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/d4135d8464eb4a0da8b9cfd3f8f7c7da 2023-07-18 03:56:17,482 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:17,482 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 03:56:17,483 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:56:17,483 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/rep_barrier 2023-07-18 03:56:17,483 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 03:56:17,484 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:17,484 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 03:56:17,484 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:56:17,484 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table 2023-07-18 03:56:17,485 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 03:56:17,491 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/table/e35dec1443364b0288f9553a65a94860 2023-07-18 03:56:17,491 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:17,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:56:17,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740 2023-07-18 03:56:17,495 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 03:56:17,496 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 03:56:17,497 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72495984, jitterRate=0.08027434349060059}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 03:56:17,497 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 03:56:17,498 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689652577445 2023-07-18 03:56:17,502 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 03:56:17,503 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 03:56:17,503 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45575,1689652576900, state=OPEN 2023-07-18 03:56:17,505 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 03:56:17,505 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 03:56:17,507 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-18 03:56:17,507 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45575,1689652576900 in 214 msec 2023-07-18 03:56:17,509 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-18 03:56:17,509 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 381 msec 2023-07-18 03:56:18,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-18 03:56:19,675 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:56:20,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-18 03:56:21,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-18 03:56:21,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 03:56:21,649 DEBUG [hconnection-0x3f958393-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:21,650 INFO [RS-EventLoopGroup-17-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44432, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:56:21,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6596a12435c5c3b7add032ab29aea733, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40667,1689652542338, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338, openSeqNum=10 2023-07-18 03:56:21,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6d104435467e1000121c02223fd064d0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40667,1689652542338, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338, openSeqNum=11 2023-07-18 03:56:21,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40667,1689652542338, regionLocation=jenkins-hbase4.apache.org,40667,1689652542338, openSeqNum=11 2023-07-18 03:56:21,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-18 03:56:21,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689652641657 2023-07-18 03:56:21,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689652701657 2023-07-18 03:56:21,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-18 03:56:21,672 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,40667,1689652542338 had 4 regions 2023-07-18 03:56:21,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40927,1689652576876-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:21,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40927,1689652576876-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:21,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40927,1689652576876-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:21,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40927, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:21,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 03:56:21,672 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. is NOT online; state={6596a12435c5c3b7add032ab29aea733 state=OPEN, ts=1689652581657, server=jenkins-hbase4.apache.org,40667,1689652542338}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-18 03:56:21,673 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,40667,1689652542338, splitWal=true, meta=true, isMeta: false 2023-07-18 03:56:21,676 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338-splitting dir is empty, no logs to split. 2023-07-18 03:56:21,676 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,40667,1689652542338 WAL count=0, meta=false 2023-07-18 03:56:21,676 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,40667,1689652542338/hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., unknown_server=jenkins-hbase4.apache.org,40667,1689652542338/t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0., unknown_server=jenkins-hbase4.apache.org,40667,1689652542338/t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:21,678 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/WALs/jenkins-hbase4.apache.org,40667,1689652542338-splitting dir is empty, no logs to split. 2023-07-18 03:56:21,678 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,40667,1689652542338 WAL count=0, meta=false 2023-07-18 03:56:21,678 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,40667,1689652542338 WAL splitting is done? wals=0, meta=false 2023-07-18 03:56:21,681 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN}] 2023-07-18 03:56:21,682 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN 2023-07-18 03:56:21,682 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN 2023-07-18 03:56:21,682 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN 2023-07-18 03:56:21,683 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:21,683 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:21,683 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 03:56:21,683 DEBUG [jenkins-hbase4:40927] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 03:56:21,685 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=6596a12435c5c3b7add032ab29aea733, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:21,685 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:21,685 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652581685"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652581685"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652581685"}]},"ts":"1689652581685"} 2023-07-18 03:56:21,685 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652581685"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652581685"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652581685"}]},"ts":"1689652581685"} 2023-07-18 03:56:21,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 6596a12435c5c3b7add032ab29aea733, server=jenkins-hbase4.apache.org,45575,1689652576900}] 2023-07-18 03:56:21,688 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=30, state=RUNNABLE; OpenRegionProcedure d59bee2bde49e7e5abb27771a5bfd3f4, server=jenkins-hbase4.apache.org,45575,1689652576900}] 2023-07-18 03:56:21,837 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=6d104435467e1000121c02223fd064d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:21,837 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652581837"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689652581837"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689652581837"}]},"ts":"1689652581837"} 2023-07-18 03:56:21,841 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=32, state=RUNNABLE; OpenRegionProcedure 6d104435467e1000121c02223fd064d0, server=jenkins-hbase4.apache.org,45575,1689652576900}] 2023-07-18 03:56:21,845 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:21,845 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6596a12435c5c3b7add032ab29aea733, NAME => 'hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:21,845 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,845 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:21,845 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,845 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,847 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:21,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d59bee2bde49e7e5abb27771a5bfd3f4, NAME => 't2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:21,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,848 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info 2023-07-18 03:56:21,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:21,848 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info 2023-07-18 03:56:21,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,848 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6596a12435c5c3b7add032ab29aea733 columnFamilyName info 2023-07-18 03:56:21,849 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,850 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1 2023-07-18 03:56:21,850 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1 2023-07-18 03:56:21,851 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d59bee2bde49e7e5abb27771a5bfd3f4 columnFamilyName cf1 2023-07-18 03:56:21,855 DEBUG [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/info/7d27b66e1d264872bc50ba77246226b1 2023-07-18 03:56:21,855 INFO [StoreOpener-6596a12435c5c3b7add032ab29aea733-1] regionserver.HStore(310): Store=6596a12435c5c3b7add032ab29aea733/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:21,856 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,857 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/3d270ab02d934440a4e65c4aa122741c 2023-07-18 03:56:21,857 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,860 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:21,861 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6596a12435c5c3b7add032ab29aea733; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61770244, jitterRate=-0.07955163717269897}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 03:56:21,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:56:21,864 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., pid=33, masterSystemTime=1689652581839 2023-07-18 03:56:21,865 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbc414ea1b3449fd9d09d2d92f427eb0 2023-07-18 03:56:21,865 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/bbc414ea1b3449fd9d09d2d92f427eb0 2023-07-18 03:56:21,866 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(310): Store=d59bee2bde49e7e5abb27771a5bfd3f4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:21,866 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,867 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:21,867 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:21,867 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/norep 2023-07-18 03:56:21,867 DEBUG [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/norep 2023-07-18 03:56:21,867 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=6596a12435c5c3b7add032ab29aea733, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:21,867 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d59bee2bde49e7e5abb27771a5bfd3f4 columnFamilyName norep 2023-07-18 03:56:21,867 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689652581867"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652581867"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652581867"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652581867"}]},"ts":"1689652581867"} 2023-07-18 03:56:21,868 INFO [StoreOpener-d59bee2bde49e7e5abb27771a5bfd3f4-1] regionserver.HStore(310): Store=d59bee2bde49e7e5abb27771a5bfd3f4/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:21,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,870 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-07-18 03:56:21,870 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 6596a12435c5c3b7add032ab29aea733, server=jenkins-hbase4.apache.org,45575,1689652576900 in 182 msec 2023-07-18 03:56:21,872 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6596a12435c5c3b7add032ab29aea733, ASSIGN in 191 msec 2023-07-18 03:56:21,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:56:21,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:21,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d59bee2bde49e7e5abb27771a5bfd3f4; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64157270, jitterRate=-0.04398217797279358}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:56:21,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:56:21,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4., pid=34, masterSystemTime=1689652581839 2023-07-18 03:56:21,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:21,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:21,877 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=d59bee2bde49e7e5abb27771a5bfd3f4, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:21,877 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652581877"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652581877"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652581877"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652581877"}]},"ts":"1689652581877"} 2023-07-18 03:56:21,880 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=30 2023-07-18 03:56:21,880 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=30, state=SUCCESS; OpenRegionProcedure d59bee2bde49e7e5abb27771a5bfd3f4, server=jenkins-hbase4.apache.org,45575,1689652576900 in 190 msec 2023-07-18 03:56:21,881 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=d59bee2bde49e7e5abb27771a5bfd3f4, ASSIGN in 201 msec 2023-07-18 03:56:21,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:21,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6d104435467e1000121c02223fd064d0, NAME => 't1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.', STARTKEY => '', ENDKEY => ''} 2023-07-18 03:56:21,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:21,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 03:56:21,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:21,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:21,997 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:21,998 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1 2023-07-18 03:56:21,998 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1 2023-07-18 03:56:21,999 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d104435467e1000121c02223fd064d0 columnFamilyName cf1 2023-07-18 03:56:22,005 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 967b742e58634ba0879b99eae13ac0ae 2023-07-18 03:56:22,005 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/967b742e58634ba0879b99eae13ac0ae 2023-07-18 03:56:22,009 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(539): loaded hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/f7b510ff27dc4e3286f94f9b58c2ef73 2023-07-18 03:56:22,010 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(310): Store=6d104435467e1000121c02223fd064d0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:22,010 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:22,011 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/norep 2023-07-18 03:56:22,011 DEBUG [StoreOpener-6d104435467e1000121c02223fd064d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/norep 2023-07-18 03:56:22,011 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d104435467e1000121c02223fd064d0 columnFamilyName norep 2023-07-18 03:56:22,012 INFO [StoreOpener-6d104435467e1000121c02223fd064d0-1] regionserver.HStore(310): Store=6d104435467e1000121c02223fd064d0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 03:56:22,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:56:22,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0 2023-07-18 03:56:22,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 03:56:22,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:22,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6d104435467e1000121c02223fd064d0; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68002398, jitterRate=0.013314694166183472}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 03:56:22,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:56:22,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0., pid=35, masterSystemTime=1689652581992 2023-07-18 03:56:22,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:22,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:22,019 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=6d104435467e1000121c02223fd064d0, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:22,019 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689652582019"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689652582019"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689652582019"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689652582019"}]},"ts":"1689652582019"} 2023-07-18 03:56:22,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=32 2023-07-18 03:56:22,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=32, state=SUCCESS; OpenRegionProcedure 6d104435467e1000121c02223fd064d0, server=jenkins-hbase4.apache.org,45575,1689652576900 in 181 msec 2023-07-18 03:56:22,023 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=27 2023-07-18 03:56:22,023 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,40667,1689652542338 after splitting done 2023-07-18 03:56:22,023 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=6d104435467e1000121c02223fd064d0, ASSIGN in 343 msec 2023-07-18 03:56:22,023 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,40667,1689652542338 from processing; numProcessing=0 2023-07-18 03:56:22,025 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,40667,1689652542338, splitWal=true, meta=true in 4.9400 sec 2023-07-18 03:56:22,202 INFO [jenkins-hbase4:45575Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:56:22,673 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-18 03:56:22,685 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-18 03:56:22,689 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-18 03:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.761sec 2023-07-18 03:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 03:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 03:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 03:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40927,1689652576876-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 03:56:22,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40927,1689652576876-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 03:56:22,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 03:56:22,725 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x0fc1791e to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:22,733 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7998881d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:22,735 DEBUG [hconnection-0x2b323576-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:22,736 INFO [RS-EventLoopGroup-17-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44438, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:56:22,739 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-18 03:56:22,739 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0fc1791e to 127.0.0.1:60939 2023-07-18 03:56:22,739 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:22,740 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:40927 after: jenkins-hbase4.apache.org:40927 2023-07-18 03:56:22,741 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(139): Connect 0x541dd40e to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:22,745 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ebed27e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:23,165 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-18 03:56:23,165 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-18 03:56:23,166 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-18 03:56:23,166 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-18 03:56:23,247 DEBUG [hconnection-0x2cea0d88-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:23,249 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44444, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 03:56:23,269 INFO [Listener at localhost/46383] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689652583269 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:23,272 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication16896525832690x0, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:23,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689652583269-0x1017691f55f0038 connected Start Replication Server start 2023-07-18 03:56:23,275 INFO [Listener at localhost/46383] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-18 03:56:23,287 INFO [Listener at localhost/46383] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 03:56:23,288 DEBUG [Listener at localhost/46383] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 03:56:23,298 INFO [Listener at localhost/46383] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:56:23,299 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,40321,1689652569096] other RSs: [] 2023-07-18 03:56:23,317 DEBUG [Listener at localhost/46383.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x5ebf829d to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:23,323 DEBUG [Listener at localhost/46383.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@26377715, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:23,323 INFO [Listener at localhost/46383.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:23,325 DEBUG [Listener at localhost/46383.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:23,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f003a connected 2023-07-18 03:56:23,326 INFO [Listener at localhost/46383.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:56:25,332 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,40321,1689652569096/1's WALs to 1689652583275.syncuptool.replication.org,1234,1 2023-07-18 03:56:25,335 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509 with data PBUF\x08\x8A\x09 2023-07-18 03:56:25,336 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,40321,1689652569096/1's WALs to 1689652583275.syncuptool.replication.org,1234,1 2023-07-18 03:56:27,202 INFO [jenkins-hbase4:45575Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:56:27,338 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:27,338 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:27,348 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:56:27,349 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:27,349 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,40321,1689652569096, ReplicationSource: 1, currentBandwidth=0 2023-07-18 03:56:27,349 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:27,369 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096] zookeeper.ReadOnlyZKClient(139): Connect 0x2791c5ba to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:27,373 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@39801c14, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:27,373 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:60939 2023-07-18 03:56:27,376 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 03:56:27,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x1017691f55f003c connected 2023-07-18 03:56:27,378 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,40321,1689652569096 (queues=1) is replicating from cluster=0fbe7ec6-8a73-4a70-9636-fde3fbaa86f3 to cluster=7bda9a11-cbe3-49e8-82ca-30b3298b646d 2023-07-18 03:56:27,378 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C40321%2C1689652569096 2023-07-18 03:56:27,384 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,40321,1689652569096, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 03:56:27,384 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C40321%2C1689652569096 2023-07-18 03:56:27,397 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509 2023-07-18 03:56:27,397 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-18 03:56:27,397 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 03:56:27,399 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,40321,1689652569096 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/f1f3df17ddf51de240de231140850f62/112=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/113=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/114=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/115=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/116=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/117=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/118=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/119=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/120=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/121=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/122=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/123=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/124=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/125=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/126=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/127=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/128=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/129=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/130=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/131=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/132=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/133=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/134=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/135=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/136=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/137=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/138=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/139=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/140=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/141=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/142=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/143=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/144=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/145=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/146=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/147=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/148=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/149=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/150=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/151=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/152=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/153=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/154=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/155=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/156=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/157=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/158=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/159=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/160=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/161=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/162=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/163=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/164=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/165=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/166=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/167=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/168=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/169=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/170=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/171=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/172=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/173=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/174=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/175=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/176=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/177=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/178=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/179=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/180=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/181=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/182=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/183=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/184=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/185=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/186=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/187=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/188=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/189=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/190=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/191=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/192=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/193=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/194=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/195=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/196=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/197=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/198=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/199=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/200=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/201=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/202=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/203=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/204=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/205=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/206=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/207=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/208=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/209=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/210=[#edits: 1 = ],208}, {t1_syncup/f1f3df17ddf51de240de231140850f62/211=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/212=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/213=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/214=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/215=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/216=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/217=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/218=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/219=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/220=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/221=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/222=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/223=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/224=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/225=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/226=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/227=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/228=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/229=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/230=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/231=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/232=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/233=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/234=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/235=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/236=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/237=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/238=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/239=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/240=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/241=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/242=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/243=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/244=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/245=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/246=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/247=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/248=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/249=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/250=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/251=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/252=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/253=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/254=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/255=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/256=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/257=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/258=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/259=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/260=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/261=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/262=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/263=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/264=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/265=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/266=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/267=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/268=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/269=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/270=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/271=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/272=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/273=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/274=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/275=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/276=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/277=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/278=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/279=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/280=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/281=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/282=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/283=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/284=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/285=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/286=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/287=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/288=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/289=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/290=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/291=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/292=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/293=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/294=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/295=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/296=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/297=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/298=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/299=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/300=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/301=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/302=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/303=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/304=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/305=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/306=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/307=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/308=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/309=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/310=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/311=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/312=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/313=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/314=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/315=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/316=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/317=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/318=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/319=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/320=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/321=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/322=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/323=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/324=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/325=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/326=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/327=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/328=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/329=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/330=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/331=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/332=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/333=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/334=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/335=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/336=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/337=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/338=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/339=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/340=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/341=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/342=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/343=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/344=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/345=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/346=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/347=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/348=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/349=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/350=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/351=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/352=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/353=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/354=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/355=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/356=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/357=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/358=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/359=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/360=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/361=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/362=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/363=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/364=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/365=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/366=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/367=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/368=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/369=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/370=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/371=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/372=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/373=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/374=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/375=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/376=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/377=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/378=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/379=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/380=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/381=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/382=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/383=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/384=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/385=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/386=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/387=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/388=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/389=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/390=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/391=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/392=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/393=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/394=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/395=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/396=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/397=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/398=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/399=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/400=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/401=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/402=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/403=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/404=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/405=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/406=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/407=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/408=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/409=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/410=[#edits: 1 = ],208}, {t2_syncup/f95930cf45f370dfce61fdb9733fd578/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:40155/user/jenkins/test-data/e8a9fdfb-be52-94fe-5751-9a6a0f71dd26/oldWALs/jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509, lastWalPosition=35352, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-18 03:56:27,406 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 03:56:27,408 INFO [RS-EventLoopGroup-17-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44446, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 03:56:27,412 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45575] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 03:56:27,412 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45575] zookeeper.ReadOnlyZKClient(139): Connect 0x5be0823c to 127.0.0.1:60939 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 03:56:27,416 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@55277730, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 03:56:27,417 DEBUG [hconnection-0x5118e94c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 03:56:27,419 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44456, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-18 03:56:27,436 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45575] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 03:56:27,437 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,40321,1689652569096] 2023-07-18 03:56:27,438 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C40321%2C1689652569096.1689652569509] 2023-07-18 03:56:27,439 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,40321,1689652569096 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 03:56:27,439 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C40321%2C1689652569096 of peer 1-jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:27,540 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:27,546 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652583269-0x1017691f55f0038, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689652583275.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,40321,1689652569096 2023-07-18 03:56:27,546 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096.replicationSource.shipperjenkins-hbase4.apache.org%2C40321%2C1689652569096,1-jenkins-hbase4.apache.org,40321,1689652569096] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,40321,1689652569096 with the following stats: Total replicated edits: 300, current progress: 2023-07-18 03:56:32,202 INFO [jenkins-hbase4:45575Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 03:56:32,202 INFO [jenkins-hbase4:45575Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-18 03:56:33,043 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 03:56:33,299 INFO [Listener at localhost/46383] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 03:56:33,403 DEBUG [Listener at localhost/46383.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f003a, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:33,405 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ebf829d to 127.0.0.1:60939 2023-07-18 03:56:33,405 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:33,607 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652583269-0x1017691f55f0038, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:33,607 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689652583269-0x1017691f55f0038, quorum=127.0.0.1:60939, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:33,622 INFO [Listener at localhost/46383] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-18 03:56:33,623 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-18 03:56:33,623 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x541dd40e to 127.0.0.1:60939 2023-07-18 03:56:33,623 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:33,623 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 03:56:33,623 DEBUG [Listener at localhost/46383] util.JVMClusterUtil(257): Found active master hash=631126526, stopped=false 2023-07-18 03:56:33,623 INFO [Listener at localhost/46383] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:33,625 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:56:33,625 INFO [Listener at localhost/46383] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 03:56:33,625 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:56:33,625 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 03:56:33,627 DEBUG [Listener at localhost/46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x43f6871b to 127.0.0.1:60939 2023-07-18 03:56:33,628 DEBUG [Listener at localhost/46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:33,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:56:33,628 INFO [Listener at localhost/46383] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45575,1689652576900' ***** 2023-07-18 03:56:33,628 INFO [Listener at localhost/46383] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 03:56:33,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 03:56:33,628 INFO [RS:0;jenkins-hbase4:45575] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 03:56:33,628 INFO [RS:0;jenkins-hbase4:45575] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 03:56:33,628 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 03:56:33,629 INFO [RS:0;jenkins-hbase4:45575] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 03:56:33,630 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(3305): Received CLOSE for 6596a12435c5c3b7add032ab29aea733 2023-07-18 03:56:33,630 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(3305): Received CLOSE for d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:33,630 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(3305): Received CLOSE for 6d104435467e1000121c02223fd064d0 2023-07-18 03:56:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6596a12435c5c3b7add032ab29aea733, disabling compactions & flushes 2023-07-18 03:56:33,631 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:33,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:33,631 DEBUG [RS:0;jenkins-hbase4:45575] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ce26c93 to 127.0.0.1:60939 2023-07-18 03:56:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:33,631 DEBUG [RS:0;jenkins-hbase4:45575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. after waiting 0 ms 2023-07-18 03:56:33,631 INFO [RS:0;jenkins-hbase4:45575] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 03:56:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:33,631 INFO [RS:0;jenkins-hbase4:45575] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 03:56:33,631 INFO [RS:0;jenkins-hbase4:45575] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 03:56:33,631 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 03:56:33,632 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 03:56:33,632 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 6596a12435c5c3b7add032ab29aea733=hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733., d59bee2bde49e7e5abb27771a5bfd3f4=t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4., 6d104435467e1000121c02223fd064d0=t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0.} 2023-07-18 03:56:33,632 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1504): Waiting on 1588230740, 6596a12435c5c3b7add032ab29aea733, 6d104435467e1000121c02223fd064d0, d59bee2bde49e7e5abb27771a5bfd3f4 2023-07-18 03:56:33,633 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 03:56:33,634 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 03:56:33,634 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 03:56:33,634 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 03:56:33,634 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 03:56:33,635 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-18 03:56:33,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/namespace/6596a12435c5c3b7add032ab29aea733/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-18 03:56:33,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:33,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6596a12435c5c3b7add032ab29aea733: 2023-07-18 03:56:33,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689652534869.6596a12435c5c3b7add032ab29aea733. 2023-07-18 03:56:33,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d59bee2bde49e7e5abb27771a5bfd3f4, disabling compactions & flushes 2023-07-18 03:56:33,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:33,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:33,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. after waiting 0 ms 2023-07-18 03:56:33,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:33,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d59bee2bde49e7e5abb27771a5bfd3f4 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-18 03:56:33,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/.tmp/cf1/b497fc3086d8401eacd7e00df0bfb40a 2023-07-18 03:56:33,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/.tmp/cf1/b497fc3086d8401eacd7e00df0bfb40a as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/b497fc3086d8401eacd7e00df0bfb40a 2023-07-18 03:56:33,688 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/cf1/b497fc3086d8401eacd7e00df0bfb40a, entries=200, sequenceid=21, filesize=12.2 K 2023-07-18 03:56:33,690 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for d59bee2bde49e7e5abb27771a5bfd3f4 in 36ms, sequenceid=21, compaction requested=true 2023-07-18 03:56:33,691 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:56:33,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t2_syncup/d59bee2bde49e7e5abb27771a5bfd3f4/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-18 03:56:33,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:33,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d59bee2bde49e7e5abb27771a5bfd3f4: 2023-07-18 03:56:33,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689652537573.d59bee2bde49e7e5abb27771a5bfd3f4. 2023-07-18 03:56:33,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6d104435467e1000121c02223fd064d0, disabling compactions & flushes 2023-07-18 03:56:33,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:33,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:33,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. after waiting 0 ms 2023-07-18 03:56:33,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:33,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6d104435467e1000121c02223fd064d0 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-18 03:56:33,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/.tmp/cf1/4bd1f56bd4364a9f871941547435700b 2023-07-18 03:56:33,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/.tmp/cf1/4bd1f56bd4364a9f871941547435700b as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/4bd1f56bd4364a9f871941547435700b 2023-07-18 03:56:33,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/cf1/4bd1f56bd4364a9f871941547435700b, entries=100, sequenceid=21, filesize=8.3 K 2023-07-18 03:56:33,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 6d104435467e1000121c02223fd064d0 in 31ms, sequenceid=21, compaction requested=true 2023-07-18 03:56:33,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/default/t1_syncup/6d104435467e1000121c02223fd064d0/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-18 03:56:33,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:33,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6d104435467e1000121c02223fd064d0: 2023-07-18 03:56:33,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689652536956.6d104435467e1000121c02223fd064d0. 2023-07-18 03:56:33,832 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-18 03:56:34,033 DEBUG [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-18 03:56:34,066 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/info/c38c55c7c6274d9daba354da9b71e2a5 2023-07-18 03:56:34,074 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/.tmp/info/c38c55c7c6274d9daba354da9b71e2a5 as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/c38c55c7c6274d9daba354da9b71e2a5 2023-07-18 03:56:34,079 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/info/c38c55c7c6274d9daba354da9b71e2a5, entries=24, sequenceid=43, filesize=7.4 K 2023-07-18 03:56:34,080 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 445ms, sequenceid=43, compaction requested=true 2023-07-18 03:56:34,097 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-18 03:56:34,097 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 03:56:34,098 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 03:56:34,098 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 03:56:34,098 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 03:56:34,233 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-18 03:56:34,233 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-18 03:56:34,233 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45575,1689652576900; all regions closed. 2023-07-18 03:56:34,245 DEBUG [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:56:34,245 INFO [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45575%2C1689652576900.meta:.meta(num 1689652577453) 2023-07-18 03:56:34,252 DEBUG [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/oldWALs 2023-07-18 03:56:34,253 INFO [RS:0;jenkins-hbase4:45575] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45575%2C1689652576900:(num 1689652577305) 2023-07-18 03:56:34,253 DEBUG [RS:0;jenkins-hbase4:45575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:34,253 INFO [RS:0;jenkins-hbase4:45575] regionserver.LeaseManager(133): Closed leases 2023-07-18 03:56:34,253 INFO [RS:0;jenkins-hbase4:45575] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-18 03:56:34,253 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:56:34,253 DEBUG [RS:0;jenkins-hbase4:45575] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5be0823c to 127.0.0.1:60939 2023-07-18 03:56:34,253 DEBUG [RS:0;jenkins-hbase4:45575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:34,254 INFO [RS:0;jenkins-hbase4:45575] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45575 2023-07-18 03:56:34,257 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0029, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:34,257 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f0026, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:34,257 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x1017691f55f003c, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:34,257 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:34,257 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:34,257 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:34,257 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45575,1689652576900 2023-07-18 03:56:34,257 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 03:56:34,257 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 03:56:34,258 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45575,1689652576900] 2023-07-18 03:56:34,258 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45575,1689652576900; numProcessing=1 2023-07-18 03:56:34,260 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45575,1689652576900 already deleted, retry=false 2023-07-18 03:56:34,260 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45575,1689652576900 expired; onlineServers=0 2023-07-18 03:56:34,260 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40927,1689652576876' ***** 2023-07-18 03:56:34,260 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 03:56:34,261 DEBUG [M:0;jenkins-hbase4:40927] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21a1a995, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 03:56:34,261 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40927,1689652576876 2023-07-18 03:56:34,261 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40927,1689652576876; all regions closed. 2023-07-18 03:56:34,261 DEBUG [M:0;jenkins-hbase4:40927] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 03:56:34,261 DEBUG [M:0;jenkins-hbase4:40927] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 03:56:34,261 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 03:56:34,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652577137] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689652577137,5,FailOnTimeoutGroup] 2023-07-18 03:56:34,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652577136] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689652577136,5,FailOnTimeoutGroup] 2023-07-18 03:56:34,261 DEBUG [M:0;jenkins-hbase4:40927] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 03:56:34,263 INFO [M:0;jenkins-hbase4:40927] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 03:56:34,263 INFO [M:0;jenkins-hbase4:40927] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 03:56:34,263 INFO [M:0;jenkins-hbase4:40927] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 03:56:34,263 DEBUG [M:0;jenkins-hbase4:40927] master.HMaster(1512): Stopping service threads 2023-07-18 03:56:34,263 INFO [M:0;jenkins-hbase4:40927] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 03:56:34,263 ERROR [M:0;jenkins-hbase4:40927] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-18 03:56:34,263 INFO [M:0;jenkins-hbase4:40927] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 03:56:34,263 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-18 03:56:34,264 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 03:56:34,264 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 03:56:34,264 DEBUG [M:0;jenkins-hbase4:40927] zookeeper.ZKUtil(398): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-18 03:56:34,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 03:56:34,264 WARN [M:0;jenkins-hbase4:40927] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 03:56:34,264 INFO [M:0;jenkins-hbase4:40927] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 03:56:34,265 INFO [M:0;jenkins-hbase4:40927] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 03:56:34,265 DEBUG [M:0;jenkins-hbase4:40927] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 03:56:34,265 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:34,265 DEBUG [M:0;jenkins-hbase4:40927] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:34,265 DEBUG [M:0;jenkins-hbase4:40927] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 03:56:34,265 DEBUG [M:0;jenkins-hbase4:40927] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:34,265 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-18 03:56:34,280 INFO [M:0;jenkins-hbase4:40927] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=310 (bloomFilter=true), to=hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/be35a654be764309a2862969c6a1558b 2023-07-18 03:56:34,286 DEBUG [M:0;jenkins-hbase4:40927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/be35a654be764309a2862969c6a1558b as hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/be35a654be764309a2862969c6a1558b 2023-07-18 03:56:34,292 INFO [M:0;jenkins-hbase4:40927] regionserver.HStore(1080): Added hdfs://localhost:36001/user/jenkins/test-data/eb477154-00f7-2f5e-075c-0f8373d459eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/be35a654be764309a2862969c6a1558b, entries=9, sequenceid=310, filesize=5.5 K 2023-07-18 03:56:34,293 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 28ms, sequenceid=310, compaction requested=true 2023-07-18 03:56:34,294 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 03:56:34,294 DEBUG [M:0;jenkins-hbase4:40927] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 03:56:34,299 INFO [M:0;jenkins-hbase4:40927] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 03:56:34,299 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 03:56:34,300 INFO [M:0;jenkins-hbase4:40927] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40927 2023-07-18 03:56:34,302 DEBUG [M:0;jenkins-hbase4:40927] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,40927,1689652576876 already deleted, retry=false 2023-07-18 03:56:34,457 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:34,457 INFO [M:0;jenkins-hbase4:40927] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40927,1689652576876; zookeeper connection closed. 2023-07-18 03:56:34,457 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): master:40927-0x1017691f55f0032, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:34,557 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:34,557 INFO [RS:0;jenkins-hbase4:45575] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45575,1689652576900; zookeeper connection closed. 2023-07-18 03:56:34,557 DEBUG [Listener at localhost/46383-EventThread] zookeeper.ZKWatcher(600): regionserver:45575-0x1017691f55f0033, quorum=127.0.0.1:60939, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 03:56:34,557 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5bb746de] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5bb746de 2023-07-18 03:56:34,558 INFO [Listener at localhost/46383] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-18 03:56:34,558 WARN [Listener at localhost/46383] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 03:56:34,564 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:34,670 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 03:56:34,670 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1012145483-172.31.14.131-1689652532356 (Datanode Uuid 1d376b4a-6efa-4b84-93e4-5173e152d9c5) service to localhost/127.0.0.1:36001 2023-07-18 03:56:34,672 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data7/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:34,672 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data8/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:34,676 WARN [Listener at localhost/46383] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 03:56:34,679 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:34,786 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 03:56:34,786 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1012145483-172.31.14.131-1689652532356 (Datanode Uuid 9fd06340-54ee-4f42-95a8-f0f5c6d6fc6d) service to localhost/127.0.0.1:36001 2023-07-18 03:56:34,786 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data5/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:34,787 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data6/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:34,789 WARN [Listener at localhost/46383] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 03:56:34,792 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:34,895 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 03:56:34,896 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1012145483-172.31.14.131-1689652532356 (Datanode Uuid fedb5cbb-ab52-44f2-bc0e-eef7cdfba526) service to localhost/127.0.0.1:36001 2023-07-18 03:56:34,897 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data3/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:34,897 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data4/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:34,899 WARN [Listener at localhost/46383] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 03:56:34,904 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:35,009 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 03:56:35,009 WARN [BP-1012145483-172.31.14.131-1689652532356 heartbeating to localhost/127.0.0.1:36001] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1012145483-172.31.14.131-1689652532356 (Datanode Uuid beba6500-c858-4d25-9c79-42e747dc8fe0) service to localhost/127.0.0.1:36001 2023-07-18 03:56:35,010 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data1/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:35,010 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7b86ff6b-00d7-90a1-89ad-f75530e9e399/cluster_1086759f-6511-ff3e-6b8f-c74b4531f542/dfs/data/data2/current/BP-1012145483-172.31.14.131-1689652532356] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:35,041 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:35,217 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-18 03:56:35,218 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-18 03:56:35,218 WARN [Listener at localhost/46383] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 03:56:35,220 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:35,322 WARN [BP-1872344512-172.31.14.131-1689652524884 heartbeating to localhost/127.0.0.1:40155] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 03:56:35,322 WARN [BP-1872344512-172.31.14.131-1689652524884 heartbeating to localhost/127.0.0.1:40155] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1872344512-172.31.14.131-1689652524884 (Datanode Uuid 18443c7a-db9c-4626-bd93-45d3820bdeb4) service to localhost/127.0.0.1:40155 2023-07-18 03:56:35,323 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/dfs/data/data3/current/BP-1872344512-172.31.14.131-1689652524884] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:35,323 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/dfs/data/data4/current/BP-1872344512-172.31.14.131-1689652524884] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:35,324 WARN [Listener at localhost/46383] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 03:56:35,326 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:35,427 WARN [BP-1872344512-172.31.14.131-1689652524884 heartbeating to localhost/127.0.0.1:40155] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 03:56:35,427 WARN [BP-1872344512-172.31.14.131-1689652524884 heartbeating to localhost/127.0.0.1:40155] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1872344512-172.31.14.131-1689652524884 (Datanode Uuid 091fa510-0dd7-4ae4-a681-f35382dac693) service to localhost/127.0.0.1:40155 2023-07-18 03:56:35,427 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/dfs/data/data1/current/BP-1872344512-172.31.14.131-1689652524884] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:35,428 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/913c1e2e-63e1-5734-311c-d3851677e5cf/cluster_63f2972b-0820-028c-0a70-dda287855dd8/dfs/data/data2/current/BP-1872344512-172.31.14.131-1689652524884] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 03:56:35,436 INFO [Listener at localhost/46383] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 03:56:35,550 INFO [Listener at localhost/46383] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-18 03:56:35,580 INFO [Listener at localhost/46383] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-18 03:56:35,602 INFO [Listener at localhost/46383] hbase.ResourceChecker(175): after: replication.TestReplicationSyncUpTool#testSyncUpTool Thread=185 (was 10) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:40155 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x2791c5ba-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x1ffdd212 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1689081991.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x1ffdd212-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x5b6d6fe7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1689081991.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x1ffdd212-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:40155 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:36001 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x7202ca53 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1689081991.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x5b6d6fe7-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x5b6d6fe7-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:40155 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x2791c5ba sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1689081991.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x460e10e6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1689081991.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: RS-EventLoopGroup-17-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x460e10e6-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x2791c5ba-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:36001 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins@localhost:36001 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,42337,1689652529544-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-15-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,36729,1689652529501-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:36001 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:40155 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1232191603) connection to localhost/127.0.0.1:40155 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@6c8d697 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:40155 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x7202ca53-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x460e10e6-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/46383 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:60939@0x7202ca53-SendThread(127.0.0.1:60939) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,40321,1689652569096-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) - Thread LEAK? -, OpenFileDescriptor=656 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 224), ProcessCount=172 (was 172), AvailableMemoryMB=6123 (was 7009)